2024-09-16T13:54:46.439 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T13:54:46.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T13:54:46.537 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read} email: sangadi@redhat.com first_in_suite: false job_id: '7907316' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: client: rbd cache: true rbd clone copy on read: true global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_sangadi@teuthology priority: 130 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9764 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_10:44:09 tube: smithi user: sangadi verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T13:54:46.537 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-16T13:54:46.538 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-16T13:54:46.538 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T13:54:46.540 INFO:teuthology.task.internal:Checking packages... 2024-09-16T13:54:46.568 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-16T13:54:46.569 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T13:54:46.569 INFO:teuthology.packaging:ref: None 2024-09-16T13:54:46.569 INFO:teuthology.packaging:tag: None 2024-09-16T13:54:46.569 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T13:54:46.569 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-16T13:54:46.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T13:54:46.724 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643-gd75f2382-1focal 2024-09-16T13:54:46.726 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T13:54:46.732 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T13:54:46.732 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T13:54:46.775 INFO:teuthology.task.internal:Saving configuration 2024-09-16T13:54:46.787 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T13:54:46.800 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T13:54:46.826 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 13:48:24.542427', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-16T13:54:46.849 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 13:48:24.540552', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-16T13:54:46.849 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T13:54:46.855 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-16T13:54:46.855 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-16T13:54:46.855 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T13:54:46.932 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6116ccbac0>, signals=[15]) 2024-09-16T13:54:46.932 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T13:54:46.937 INFO:teuthology.task.internal:Opening connections... 2024-09-16T13:54:46.937 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T13:54:46.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:54:47.033 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T13:54:47.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:54:47.122 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T13:54:47.128 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-09-16T13:54:47.135 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-09-16T13:54:47.135 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Ubuntu" 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:ID=ubuntu 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE=debian 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="20.04" 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T13:54:47.185 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T13:54:47.186 INFO:teuthology.orchestra.run.smithi079.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T13:54:47.186 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_CODENAME=focal 2024-09-16T13:54:47.186 INFO:teuthology.orchestra.run.smithi079.stdout:UBUNTU_CODENAME=focal 2024-09-16T13:54:47.186 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-09-16T13:54:47.213 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-16T13:54:47.220 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-16T13:54:47.220 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-16T13:54:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Ubuntu" 2024-09-16T13:54:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-16T13:54:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:ID=ubuntu 2024-09-16T13:54:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE=debian 2024-09-16T13:54:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-16T13:54:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="20.04" 2024-09-16T13:54:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T13:54:47.269 INFO:teuthology.orchestra.run.smithi164.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T13:54:47.270 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T13:54:47.270 INFO:teuthology.orchestra.run.smithi164.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T13:54:47.270 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_CODENAME=focal 2024-09-16T13:54:47.270 INFO:teuthology.orchestra.run.smithi164.stdout:UBUNTU_CODENAME=focal 2024-09-16T13:54:47.270 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-16T13:54:47.312 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T13:54:47.320 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T13:54:47.341 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T13:54:47.341 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-09-16T13:54:47.344 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-16T13:54:47.349 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T13:54:47.366 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T13:54:47.366 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-09-16T13:54:47.369 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-16T13:54:47.389 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T13:54:47.444 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T13:54:47.458 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T13:54:47.458 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T13:54:47.458 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T13:54:47.459 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T13:54:47.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T13:54:47.464 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-16T13:54:47.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T13:54:47.465 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-16T13:54:47.471 INFO:teuthology.orchestra.run.smithi079.stdout:5.4.0-124-generic 2024-09-16T13:54:47.472 INFO:teuthology.task.kernel:Running kernel on smithi079: 5.4.0-124-generic 2024-09-16T13:54:47.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get clean 2024-09-16T13:54:47.473 INFO:teuthology.orchestra.run.smithi164.stdout:5.4.0-124-generic 2024-09-16T13:54:47.473 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.4.0-124-generic 2024-09-16T13:54:47.473 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get clean 2024-09-16T13:54:47.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get update 2024-09-16T13:54:47.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-09-16T13:54:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T13:54:47.748 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-16T13:54:47.816 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-16T13:54:47.852 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-16T13:54:47.878 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T13:54:47.878 INFO:teuthology.orchestra.run.smithi079.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-16T13:54:47.908 INFO:teuthology.orchestra.run.smithi079.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-16T13:54:48.141 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,566 kB] 2024-09-16T13:54:48.261 INFO:teuthology.orchestra.run.smithi079.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-16T13:54:48.279 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-16T13:54:48.291 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-16T13:54:48.297 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-16T13:54:48.298 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,240 kB] 2024-09-16T13:54:48.334 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-16T13:54:48.334 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [454 kB] 2024-09-16T13:54:48.338 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T13:54:48.338 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-16T13:54:48.338 INFO:teuthology.orchestra.run.smithi079.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-16T13:54:48.344 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-16T13:54:48.353 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-16T13:54:48.355 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-16T13:54:48.355 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-16T13:54:48.356 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-16T13:54:48.357 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-16T13:54:48.357 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-16T13:54:48.369 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-16T13:54:48.370 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-16T13:54:48.370 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-16T13:54:48.387 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-16T13:54:48.387 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-16T13:54:48.387 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-16T13:54:48.462 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-16T13:54:48.561 INFO:teuthology.orchestra.run.smithi079.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-16T13:54:48.691 INFO:teuthology.orchestra.run.smithi079.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-16T13:54:48.797 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-16T13:54:48.883 INFO:teuthology.orchestra.run.smithi079.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-16T13:54:48.898 INFO:teuthology.orchestra.run.smithi079.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-16T13:54:48.898 INFO:teuthology.orchestra.run.smithi079.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-16T13:54:48.987 INFO:teuthology.orchestra.run.smithi079.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-16T13:54:48.992 INFO:teuthology.orchestra.run.smithi079.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-16T13:54:48.996 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-16T13:54:49.002 INFO:teuthology.orchestra.run.smithi079.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T13:54:49.002 INFO:teuthology.orchestra.run.smithi079.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-16T13:54:49.012 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-16T13:54:49.012 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-16T13:54:49.021 INFO:teuthology.orchestra.run.smithi079.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-16T13:54:49.051 INFO:teuthology.orchestra.run.smithi079.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-16T13:54:49.058 INFO:teuthology.orchestra.run.smithi079.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-16T13:54:49.059 INFO:teuthology.orchestra.run.smithi079.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-16T13:54:49.059 INFO:teuthology.orchestra.run.smithi079.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-16T13:54:49.106 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-16T13:54:49.106 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-16T13:54:49.116 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T13:54:49.116 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-16T13:54:49.131 INFO:teuthology.orchestra.run.smithi079.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-16T13:54:49.131 INFO:teuthology.orchestra.run.smithi079.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-16T13:54:49.134 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-16T13:54:49.165 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-16T13:54:49.168 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-16T13:54:49.172 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-16T13:54:49.173 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-16T13:54:49.178 INFO:teuthology.orchestra.run.smithi079.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,566 kB] 2024-09-16T13:54:49.246 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-16T13:54:49.246 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-16T13:54:49.449 INFO:teuthology.orchestra.run.smithi079.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-16T13:54:49.539 INFO:teuthology.orchestra.run.smithi079.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-16T13:54:49.622 INFO:teuthology.orchestra.run.smithi079.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-16T13:54:49.706 INFO:teuthology.orchestra.run.smithi079.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,240 kB] 2024-09-16T13:54:49.876 INFO:teuthology.orchestra.run.smithi079.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [454 kB] 2024-09-16T13:54:49.966 INFO:teuthology.orchestra.run.smithi079.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T13:54:50.049 INFO:teuthology.orchestra.run.smithi079.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-16T13:54:50.147 INFO:teuthology.orchestra.run.smithi079.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-16T13:54:50.242 INFO:teuthology.orchestra.run.smithi079.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-16T13:54:50.328 INFO:teuthology.orchestra.run.smithi079.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-16T13:54:50.411 INFO:teuthology.orchestra.run.smithi079.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-16T13:54:50.493 INFO:teuthology.orchestra.run.smithi079.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-16T13:54:50.576 INFO:teuthology.orchestra.run.smithi079.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-16T13:54:50.785 INFO:teuthology.orchestra.run.smithi079.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-16T13:54:50.868 INFO:teuthology.orchestra.run.smithi079.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-16T13:54:50.951 INFO:teuthology.orchestra.run.smithi079.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-16T13:54:51.033 INFO:teuthology.orchestra.run.smithi079.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-16T13:54:51.116 INFO:teuthology.orchestra.run.smithi079.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-16T13:54:51.198 INFO:teuthology.orchestra.run.smithi079.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-16T13:54:51.281 INFO:teuthology.orchestra.run.smithi079.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-16T13:54:52.705 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 21.9 MB in 3s (7,128 kB/s) 2024-09-16T13:54:53.246 INFO:teuthology.orchestra.run.smithi079.stdout:Fetched 21.9 MB in 4s (6,003 kB/s) 2024-09-16T13:54:54.339 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-16T13:54:54.366 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T13:54:54.438 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-16T13:54:54.656 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-09-16T13:54:54.657 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-09-16T13:54:54.771 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-16T13:54:54.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T13:54:54.857 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-09-16T13:54:54.857 INFO:teuthology.orchestra.run.smithi164.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T13:54:54.857 INFO:teuthology.orchestra.run.smithi164.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-16T13:54:54.860 INFO:teuthology.orchestra.run.smithi164.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-16T13:54:54.860 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-09-16T13:54:54.860 INFO:teuthology.orchestra.run.smithi164.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-16T13:54:54.860 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2024-09-16T13:54:54.860 INFO:teuthology.orchestra.run.smithi164.stdout: thermald 2024-09-16T13:54:54.876 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-16T13:54:54.924 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-09-16T13:54:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T13:54:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-16T13:54:54.924 INFO:teuthology.orchestra.run.smithi164.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-16T13:54:54.926 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2024-09-16T13:54:54.926 INFO:teuthology.orchestra.run.smithi164.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T13:54:55.076 INFO:teuthology.orchestra.run.smithi079.stdout:Building dependency tree... 2024-09-16T13:54:55.076 INFO:teuthology.orchestra.run.smithi079.stdout:Reading state information... 2024-09-16T13:54:55.106 INFO:teuthology.orchestra.run.smithi164.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T13:54:55.106 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 77.2 MB of archives. 2024-09-16T13:54:55.106 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-16T13:54:55.106 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-16T13:54:55.275 INFO:teuthology.orchestra.run.smithi079.stdout:The following additional packages will be installed: 2024-09-16T13:54:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T13:54:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-16T13:54:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-16T13:54:55.277 INFO:teuthology.orchestra.run.smithi079.stdout:Suggested packages: 2024-09-16T13:54:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-16T13:54:55.277 INFO:teuthology.orchestra.run.smithi079.stdout:Recommended packages: 2024-09-16T13:54:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: thermald 2024-09-16T13:54:55.336 INFO:teuthology.orchestra.run.smithi079.stdout:The following NEW packages will be installed: 2024-09-16T13:54:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T13:54:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-16T13:54:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-16T13:54:55.339 INFO:teuthology.orchestra.run.smithi079.stdout:The following packages will be upgraded: 2024-09-16T13:54:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T13:54:55.521 INFO:teuthology.orchestra.run.smithi079.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T13:54:55.521 INFO:teuthology.orchestra.run.smithi079.stdout:Need to get 77.2 MB of archives. 2024-09-16T13:54:55.522 INFO:teuthology.orchestra.run.smithi079.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-16T13:54:55.522 INFO:teuthology.orchestra.run.smithi079.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-16T13:54:56.265 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-16T13:54:56.537 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-16T13:54:56.834 INFO:teuthology.orchestra.run.smithi079.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-16T13:54:57.107 INFO:teuthology.orchestra.run.smithi079.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-16T13:54:57.860 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-16T13:54:57.927 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-16T13:54:57.928 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-16T13:54:58.199 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-16T13:54:58.267 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-16T13:54:58.473 INFO:teuthology.orchestra.run.smithi079.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-16T13:54:58.473 INFO:teuthology.orchestra.run.smithi079.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-16T13:54:58.474 INFO:teuthology.orchestra.run.smithi079.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-16T13:54:58.751 INFO:teuthology.orchestra.run.smithi079.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-16T13:54:58.814 INFO:teuthology.orchestra.run.smithi079.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-16T13:54:59.016 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 77.2 MB in 3s (23.1 MB/s) 2024-09-16T13:54:59.215 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-16T13:54:59.491 INFO:teuthology.orchestra.run.smithi079.stdout:Fetched 77.2 MB in 3s (22.3 MB/s) 2024-09-16T13:54:59.677 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-16T13:55:00.993 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-16T13:55:00.998 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T13:55:01.162 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:01.437 INFO:teuthology.orchestra.run.smithi079.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-16T13:55:01.441 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T13:55:01.517 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:03.187 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-16T13:55:03.200 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T13:55:03.288 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:03.561 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-16T13:55:03.577 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T13:55:03.658 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:04.381 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-16T13:55:04.395 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T13:55:04.431 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:04.756 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-16T13:55:04.770 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T13:55:04.814 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:09.466 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T13:55:09.592 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T13:55:09.712 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T13:55:09.860 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T13:55:09.935 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T13:55:10.054 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T13:55:10.210 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T13:55:10.355 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T13:55:10.391 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-16T13:55:10.404 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-16T13:55:10.449 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T13:55:10.725 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-16T13:55:10.740 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-16T13:55:10.808 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T13:55:15.759 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-16T13:55:15.781 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T13:55:15.825 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:16.325 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-16T13:55:16.348 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T13:55:16.391 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:18.428 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T13:55:18.549 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T13:55:18.895 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:19.439 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T13:55:19.550 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T13:55:19.910 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:20.646 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:21.172 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:21.807 INFO:teuthology.orchestra.run.smithi079.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-16T13:55:21.807 INFO:teuthology.orchestra.run.smithi079.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-16T13:55:21.939 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:22.333 INFO:teuthology.orchestra.run.smithi164.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-16T13:55:22.333 INFO:teuthology.orchestra.run.smithi164.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-16T13:55:22.465 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:23.116 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T13:55:23.242 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-16T13:55:23.368 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:23.494 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-16T13:55:23.620 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-16T13:55:23.746 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:23.837 INFO:teuthology.orchestra.run.smithi079.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T13:55:23.837 INFO:teuthology.orchestra.run.smithi079.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-16T13:55:24.709 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T13:55:25.443 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-16T13:55:25.569 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:25.729 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-16T13:55:25.880 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-16T13:55:26.006 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T13:55:26.083 INFO:teuthology.orchestra.run.smithi164.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T13:55:26.083 INFO:teuthology.orchestra.run.smithi164.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-16T13:55:45.602 INFO:teuthology.orchestra.run.smithi079.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T13:55:45.602 INFO:teuthology.orchestra.run.smithi079.stdout:Sourcing file `/etc/default/grub' 2024-09-16T13:55:45.626 INFO:teuthology.orchestra.run.smithi079.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T13:55:45.646 INFO:teuthology.orchestra.run.smithi079.stdout:Generating grub configuration file ... 2024-09-16T13:55:45.946 INFO:teuthology.orchestra.run.smithi079.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T13:55:45.971 INFO:teuthology.orchestra.run.smithi079.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T13:55:46.337 INFO:teuthology.orchestra.run.smithi079.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T13:55:46.349 INFO:teuthology.orchestra.run.smithi079.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T13:55:48.190 INFO:teuthology.orchestra.run.smithi079.stdout:done 2024-09-16T13:55:48.379 INFO:teuthology.orchestra.run.smithi164.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T13:55:48.379 INFO:teuthology.orchestra.run.smithi164.stdout:Sourcing file `/etc/default/grub' 2024-09-16T13:55:48.403 INFO:teuthology.orchestra.run.smithi164.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T13:55:48.418 INFO:teuthology.orchestra.run.smithi164.stdout:Generating grub configuration file ... 2024-09-16T13:55:48.587 DEBUG:teuthology.orchestra.run.smithi079:> dpkg -s linux-image-generic 2024-09-16T13:55:48.636 INFO:teuthology.orchestra.run.smithi079.stdout:Package: linux-image-generic 2024-09-16T13:55:48.636 INFO:teuthology.orchestra.run.smithi079.stdout:Status: install ok installed 2024-09-16T13:55:48.636 INFO:teuthology.orchestra.run.smithi079.stdout:Priority: optional 2024-09-16T13:55:48.636 INFO:teuthology.orchestra.run.smithi079.stdout:Section: kernel 2024-09-16T13:55:48.636 INFO:teuthology.orchestra.run.smithi079.stdout:Installed-Size: 20 2024-09-16T13:55:48.636 INFO:teuthology.orchestra.run.smithi079.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T13:55:48.636 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: amd64 2024-09-16T13:55:48.637 INFO:teuthology.orchestra.run.smithi079.stdout:Source: linux-meta 2024-09-16T13:55:48.637 INFO:teuthology.orchestra.run.smithi079.stdout:Version: 5.4.0.195.193 2024-09-16T13:55:48.637 INFO:teuthology.orchestra.run.smithi079.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T13:55:48.637 INFO:teuthology.orchestra.run.smithi079.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T13:55:48.637 INFO:teuthology.orchestra.run.smithi079.stdout:Recommends: thermald 2024-09-16T13:55:48.637 INFO:teuthology.orchestra.run.smithi079.stdout:Description: Generic Linux kernel image 2024-09-16T13:55:48.637 INFO:teuthology.orchestra.run.smithi079.stdout: This package will always depend on the latest generic kernel image 2024-09-16T13:55:48.637 INFO:teuthology.orchestra.run.smithi079.stdout: available. 2024-09-16T13:55:48.637 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-16T13:55:48.637 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T13:55:48.638 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T13:55:48.638 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T13:55:48.638 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T13:55:48.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get clean 2024-09-16T13:55:48.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get update 2024-09-16T13:55:48.806 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T13:55:48.826 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T13:55:48.852 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T13:55:48.969 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T13:55:49.051 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T13:55:49.132 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T13:55:49.246 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T13:55:49.257 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T13:55:50.728 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-16T13:55:50.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T13:55:50.827 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-16T13:55:51.039 INFO:teuthology.orchestra.run.smithi079.stdout:Building dependency tree... 2024-09-16T13:55:51.040 INFO:teuthology.orchestra.run.smithi079.stdout:Reading state information... 2024-09-16T13:55:51.122 INFO:teuthology.orchestra.run.smithi164.stdout:done 2024-09-16T13:55:51.269 INFO:teuthology.orchestra.run.smithi079.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-16T13:55:51.269 INFO:teuthology.orchestra.run.smithi079.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T13:55:51.271 DEBUG:teuthology.orchestra.run.smithi079:> dpkg -s linux-image-generic 2024-09-16T13:55:51.288 INFO:teuthology.orchestra.run.smithi079.stdout:Package: linux-image-generic 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Status: install ok installed 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Priority: optional 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Section: kernel 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Installed-Size: 20 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: amd64 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Source: linux-meta 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Version: 5.4.0.195.193 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Recommends: thermald 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout:Description: Generic Linux kernel image 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: This package will always depend on the latest generic kernel image 2024-09-16T13:55:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: available. 2024-09-16T13:55:51.290 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-16T13:55:51.295 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.V96zqFl738 2024-09-16T13:55:51.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub/grub.cfg /tmp/tmp.V96zqFl738 2024-09-16T13:55:51.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.V96zqFl738 2024-09-16T13:55:51.386 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.V96zqFl738 is 9KB 2024-09-16T13:55:51.400 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.V96zqFl738 2024-09-16T13:55:51.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T13:55:51.459 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T13:55:51.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T13:55:51.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T13:55:51.493 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-16T13:55:51.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo update-grub 2024-09-16T13:55:51.606 DEBUG:teuthology.orchestra.run.smithi164:> dpkg -s linux-image-generic 2024-09-16T13:55:51.646 INFO:teuthology.orchestra.run.smithi164.stdout:Package: linux-image-generic 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Status: install ok installed 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Priority: optional 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Section: kernel 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Installed-Size: 20 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: amd64 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Source: linux-meta 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Version: 5.4.0.195.193 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Recommends: thermald 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout:Description: Generic Linux kernel image 2024-09-16T13:55:51.647 INFO:teuthology.orchestra.run.smithi164.stdout: This package will always depend on the latest generic kernel image 2024-09-16T13:55:51.648 INFO:teuthology.orchestra.run.smithi164.stdout: available. 2024-09-16T13:55:51.648 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-16T13:55:51.648 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T13:55:51.648 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T13:55:51.648 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T13:55:51.648 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T13:55:51.648 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get clean 2024-09-16T13:55:51.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-09-16T13:55:51.864 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T13:55:51.864 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T13:55:51.870 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T13:55:51.887 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T13:55:52.865 INFO:teuthology.orchestra.run.smithi079.stderr:Sourcing file `/etc/default/grub' 2024-09-16T13:55:52.875 INFO:teuthology.orchestra.run.smithi079.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T13:55:52.895 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2024-09-16T13:55:53.126 INFO:teuthology.orchestra.run.smithi079.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T13:55:53.138 INFO:teuthology.orchestra.run.smithi079.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T13:55:53.484 INFO:teuthology.orchestra.run.smithi079.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T13:55:53.496 INFO:teuthology.orchestra.run.smithi079.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T13:55:53.782 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-16T13:55:53.809 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T13:55:53.877 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-16T13:55:54.071 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-09-16T13:55:54.071 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-09-16T13:55:54.296 INFO:teuthology.orchestra.run.smithi079.stderr:done 2024-09-16T13:55:54.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2024-09-16T13:55:54.301 INFO:teuthology.orchestra.run.smithi164.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-16T13:55:54.301 INFO:teuthology.orchestra.run.smithi164.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T13:55:54.303 DEBUG:teuthology.orchestra.run.smithi164:> dpkg -s linux-image-generic 2024-09-16T13:55:54.329 INFO:teuthology.orchestra.run.smithi164.stdout:Package: linux-image-generic 2024-09-16T13:55:54.329 INFO:teuthology.orchestra.run.smithi164.stdout:Status: install ok installed 2024-09-16T13:55:54.329 INFO:teuthology.orchestra.run.smithi164.stdout:Priority: optional 2024-09-16T13:55:54.329 INFO:teuthology.orchestra.run.smithi164.stdout:Section: kernel 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Installed-Size: 20 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: amd64 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Source: linux-meta 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Version: 5.4.0.195.193 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Recommends: thermald 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout:Description: Generic Linux kernel image 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout: This package will always depend on the latest generic kernel image 2024-09-16T13:55:54.330 INFO:teuthology.orchestra.run.smithi164.stdout: available. 2024-09-16T13:55:54.331 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-16T13:55:54.336 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.oAQygG2Lcu 2024-09-16T13:55:54.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub/grub.cfg /tmp/tmp.oAQygG2Lcu 2024-09-16T13:55:54.351 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.oAQygG2Lcu 2024-09-16T13:55:54.438 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.oAQygG2Lcu is 9KB 2024-09-16T13:55:54.452 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.oAQygG2Lcu 2024-09-16T13:55:54.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T13:55:54.514 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T13:55:54.514 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T13:55:54.514 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T13:55:54.546 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-16T13:55:54.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo update-grub 2024-09-16T13:55:55.957 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub' 2024-09-16T13:55:55.969 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T13:55:55.980 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-09-16T13:55:56.320 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T13:55:56.332 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T13:55:56.782 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T13:55:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T13:55:57.657 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-09-16T13:55:57.659 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-09-16T13:56:24.314 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T13:56:24.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-16T13:56:24.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:56:27.664 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T13:56:27.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T13:56:27.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:56:42.685 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-16T13:56:46.013 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-16T13:56:51.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-16T13:56:51.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:56:54.749 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-09-16T13:56:55.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T13:56:55.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:56:58.077 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-16T13:57:06.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-16T13:57:06.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:57:10.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T13:57:10.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:57:38.507 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-16T13:57:39.053 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-16T13:57:39.053 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-16T13:57:39.053 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-09-16T13:57:39.061 INFO:teuthology.orchestra.run.smithi079.stdout:5.4.0-195-generic 2024-09-16T13:57:39.061 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-16T13:57:39.061 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T13:57:39.061 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T13:57:40.062 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T13:57:40.063 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T13:57:40.156 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2024-09-16T13:57:40.180 DEBUG:teuthology.parallel:result is None 2024-09-16T13:58:10.082 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T13:58:25.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T13:58:25.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T13:58:25.335 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-16T13:58:25.907 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T13:58:25.907 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-16T13:58:25.908 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-16T13:58:25.914 INFO:teuthology.orchestra.run.smithi164.stdout:5.4.0-195-generic 2024-09-16T13:58:25.914 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-16T13:58:25.914 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T13:58:25.914 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T13:58:26.915 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T13:58:26.915 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T13:58:26.991 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-09-16T13:58:27.024 DEBUG:teuthology.parallel:result is None 2024-09-16T13:58:27.024 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T13:58:27.032 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T13:58:27.032 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T13:58:27.035 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T13:58:27.042 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T13:58:27.071 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T13:58:27.116 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T13:58:27.116 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T13:58:27.119 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T13:58:27.136 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T13:58:27.153 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T13:58:27.154 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T13:58:27.175 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T13:58:27.193 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T13:58:27.195 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T13:58:27.204 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T13:58:27.206 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T13:58:27.208 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T13:58:27.215 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T13:58:27.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T13:58:27.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T13:58:27.233 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T13:58:27.257 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T13:58:27.258 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T13:58:27.260 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T13:58:27.266 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T13:58:27.312 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T13:58:27.322 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T13:58:27.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T13:58:27.349 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T13:58:27.356 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T13:58:27.362 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T13:58:27.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T13:58:27.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-09-16T13:58:27.391 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-16T13:58:27.483 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T13:58:27.492 INFO:teuthology.task.internal:Starting timer... 2024-09-16T13:58:27.492 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T13:58:27.525 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T13:58:27.572 DEBUG:teuthology.task.selinux:Excluding smithi079: OS 'ubuntu' does not support SELinux 2024-09-16T13:58:27.572 DEBUG:teuthology.task.selinux:Excluding smithi164: OS 'ubuntu' does not support SELinux 2024-09-16T13:58:27.572 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T13:58:27.572 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-16T13:58:27.572 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T13:58:27.573 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T13:58:27.619 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T13:58:27.633 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T13:58:27.634 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi079.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T14:04:01.309 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-09-16T14:04:01.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-16T14:04:01.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:04:01.403 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-09-16T14:04:01.527 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-09-16T14:04:01.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T14:04:01.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:04:01.617 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-16T14:04:01.738 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T14:04:01.739 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T14:04:01.751 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T14:04:01.752 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:04:01.752 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:04:01.755 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:04:01.755 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:04:01.787 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-16T14:04:01.787 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: Command line: ntpd -gq 2024-09-16T14:04:01.788 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: proto: precision = 0.064 usec (-24) 2024-09-16T14:04:01.788 INFO:teuthology.orchestra.run.smithi079.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T14:04:01.788 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T14:04:01.789 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T14:04:01.789 INFO:teuthology.orchestra.run.smithi079.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T14:04:01.790 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T14:04:01.790 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T14:04:01.790 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T14:04:01.790 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: Listen normally on 3 enp3s0f1 172.21.15.79:123 2024-09-16T14:04:01.790 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: Listen normally on 4 lo [::1]:123 2024-09-16T14:04:01.790 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce87%5]:123 2024-09-16T14:04:01.790 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:01 ntpd[13696]: Listening on routing socket on fd #22 for interface updates 2024-09-16T14:04:01.811 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-16T14:04:01.811 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: Command line: ntpd -gq 2024-09-16T14:04:01.812 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: proto: precision = 0.074 usec (-24) 2024-09-16T14:04:01.812 INFO:teuthology.orchestra.run.smithi164.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T14:04:01.812 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T14:04:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T14:04:01.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T14:04:01.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T14:04:01.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T14:04:01.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T14:04:01.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: Listen normally on 3 enp3s0f1 172.21.15.164:123 2024-09-16T14:04:01.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: Listen normally on 4 lo [::1]:123 2024-09-16T14:04:01.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:73f3%5]:123 2024-09-16T14:04:01.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:01 ntpd[13747]: Listening on routing socket on fd #22 for interface updates 2024-09-16T14:04:02.789 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.789 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.789 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.789 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:02.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:02 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:02.812 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.812 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.812 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:02.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:02 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:04.789 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:04.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:04.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:04 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:04.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:04 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:05.811 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:05 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:05.812 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:05 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:06.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:06.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:06.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:06.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:06.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:06 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:06.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:06.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:06 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:08.789 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.789 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:08.790 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:08.791 INFO:teuthology.orchestra.run.smithi079.stdout:16 Sep 14:04:08 ntpd[13696]: ntpd: time slew +0.000167 s 2024-09-16T14:04:08.791 INFO:teuthology.orchestra.run.smithi079.stdout:ntpd: time slew +0.000167s 2024-09-16T14:04:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T14:04:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:16 Sep 14:04:08 ntpd[13696]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T14:04:08.812 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T14:04:08.813 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T14:04:08.814 INFO:teuthology.orchestra.run.smithi164.stdout:16 Sep 14:04:08 ntpd[13747]: ntpd: time slew +0.001203 s 2024-09-16T14:04:08.814 INFO:teuthology.orchestra.run.smithi164.stdout:ntpd: time slew +0.001203s 2024-09-16T14:04:08.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T14:04:08.814 INFO:teuthology.orchestra.run.smithi164.stderr:16 Sep 14:04:08 ntpd[13747]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T14:04:08.856 INFO:teuthology.orchestra.run.smithi079.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T14:04:08.856 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================== 2024-09-16T14:04:08.856 INFO:teuthology.orchestra.run.smithi079.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:04:08.856 INFO:teuthology.orchestra.run.smithi079.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:04:08.856 INFO:teuthology.orchestra.run.smithi079.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:04:08.856 INFO:teuthology.orchestra.run.smithi079.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:04:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T14:04:08.884 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2024-09-16T14:04:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:04:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:04:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:04:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:04:08.884 INFO:teuthology.run_tasks:Running task install... 2024-09-16T14:04:08.892 DEBUG:teuthology.task.install:project ceph 2024-09-16T14:04:08.892 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T14:04:08.893 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-16T14:04:08.893 INFO:teuthology.task.install:Using flavor: default 2024-09-16T14:04:08.899 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T14:04:08.899 INFO:teuthology.task.install:extra packages: [] 2024-09-16T14:04:08.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-key list | grep Ceph 2024-09-16T14:04:08.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-key list | grep Ceph 2024-09-16T14:04:08.994 INFO:teuthology.orchestra.run.smithi164.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-16T14:04:08.995 INFO:teuthology.orchestra.run.smithi079.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-16T14:04:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T14:04:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T14:04:09.056 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T14:04:09.057 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T14:04:09.066 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T14:04:09.067 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T14:04:09.068 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T14:04:09.068 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T14:04:09.200 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-16T14:04:09.201 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-16T14:04:09.210 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-16T14:04:09.210 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-16T14:04:09.311 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:04:09.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T14:04:09.320 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:04:09.320 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T14:04:09.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo apt-get update 2024-09-16T14:04:09.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-09-16T14:04:09.468 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T14:04:09.524 INFO:teuthology.orchestra.run.smithi079.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-16T14:04:09.530 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T14:04:09.546 INFO:teuthology.orchestra.run.smithi079.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-16T14:04:09.570 INFO:teuthology.orchestra.run.smithi079.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-16T14:04:09.586 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T14:04:09.589 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-16T14:04:09.593 INFO:teuthology.orchestra.run.smithi079.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-16T14:04:09.612 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-16T14:04:09.627 INFO:teuthology.orchestra.run.smithi079.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-16T14:04:09.637 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-16T14:04:09.648 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T14:04:09.659 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-16T14:04:09.670 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T14:04:09.698 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-16T14:04:09.730 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T14:04:09.756 INFO:teuthology.orchestra.run.smithi079.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T14:04:09.813 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T14:04:10.124 INFO:teuthology.orchestra.run.smithi079.stdout:Fetched 28.5 kB in 1s (45.0 kB/s) 2024-09-16T14:04:10.202 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 28.5 kB in 1s (44.0 kB/s) 2024-09-16T14:04:11.281 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-16T14:04:11.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-16T14:04:11.363 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-16T14:04:11.379 INFO:teuthology.orchestra.run.smithi079.stdout:Reading package lists... 2024-09-16T14:04:11.388 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-16T14:04:11.458 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-09-16T14:04:11.597 INFO:teuthology.orchestra.run.smithi079.stdout:Building dependency tree... 2024-09-16T14:04:11.597 INFO:teuthology.orchestra.run.smithi079.stdout:Reading state information... 2024-09-16T14:04:11.653 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-09-16T14:04:11.654 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-09-16T14:04:11.791 INFO:teuthology.orchestra.run.smithi079.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T14:04:11.792 INFO:teuthology.orchestra.run.smithi079.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-16T14:04:11.793 INFO:teuthology.orchestra.run.smithi079.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T14:04:11.793 INFO:teuthology.orchestra.run.smithi079.stdout:The following additional packages will be installed: 2024-09-16T14:04:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T14:04:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-16T14:04:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-16T14:04:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-16T14:04:11.793 INFO:teuthology.orchestra.run.smithi079.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-16T14:04:11.795 INFO:teuthology.orchestra.run.smithi079.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: socat xmlstarlet 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout:Suggested packages: 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-16T14:04:11.796 INFO:teuthology.orchestra.run.smithi079.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout:Recommended packages: 2024-09-16T14:04:11.797 INFO:teuthology.orchestra.run.smithi079.stdout: btrfs-tools 2024-09-16T14:04:11.847 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-16T14:04:11.850 INFO:teuthology.orchestra.run.smithi164.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-16T14:04:11.851 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T14:04:11.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-16T14:04:11.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-16T14:04:11.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-16T14:04:11.852 INFO:teuthology.orchestra.run.smithi164.stdout: socat xmlstarlet 2024-09-16T14:04:11.852 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-09-16T14:04:11.852 INFO:teuthology.orchestra.run.smithi164.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2024-09-16T14:04:11.853 INFO:teuthology.orchestra.run.smithi164.stdout: btrfs-tools 2024-09-16T14:04:11.912 INFO:teuthology.orchestra.run.smithi079.stdout:The following NEW packages will be installed: 2024-09-16T14:04:11.912 INFO:teuthology.orchestra.run.smithi079.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T14:04:11.913 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T14:04:11.913 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-16T14:04:11.913 INFO:teuthology.orchestra.run.smithi079.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-16T14:04:11.913 INFO:teuthology.orchestra.run.smithi079.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-16T14:04:11.913 INFO:teuthology.orchestra.run.smithi079.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-16T14:04:11.913 INFO:teuthology.orchestra.run.smithi079.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-16T14:04:11.915 INFO:teuthology.orchestra.run.smithi079.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T14:04:11.916 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-16T14:04:11.916 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-16T14:04:11.916 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-16T14:04:11.916 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-16T14:04:11.916 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-16T14:04:11.916 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-16T14:04:11.916 INFO:teuthology.orchestra.run.smithi079.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-16T14:04:11.916 INFO:teuthology.orchestra.run.smithi079.stdout:The following packages will be upgraded: 2024-09-16T14:04:11.917 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 librbd1 2024-09-16T14:04:11.971 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-09-16T14:04:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T14:04:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T14:04:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-16T14:04:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-16T14:04:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-16T14:04:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-16T14:04:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-16T14:04:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2024-09-16T14:04:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 librbd1 2024-09-16T14:04:12.065 INFO:teuthology.orchestra.run.smithi079.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-16T14:04:12.065 INFO:teuthology.orchestra.run.smithi079.stdout:Need to get 201 MB of archives. 2024-09-16T14:04:12.065 INFO:teuthology.orchestra.run.smithi079.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-16T14:04:12.065 INFO:teuthology.orchestra.run.smithi079.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-16T14:04:12.109 INFO:teuthology.orchestra.run.smithi079.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-16T14:04:12.125 INFO:teuthology.orchestra.run.smithi164.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-16T14:04:12.126 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 201 MB of archives. 2024-09-16T14:04:12.126 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-16T14:04:12.126 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-16T14:04:12.170 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-16T14:04:12.245 INFO:teuthology.orchestra.run.smithi079.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-16T14:04:12.322 INFO:teuthology.orchestra.run.smithi079.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-16T14:04:12.334 INFO:teuthology.orchestra.run.smithi079.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-16T14:04:12.340 INFO:teuthology.orchestra.run.smithi079.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-16T14:04:12.341 INFO:teuthology.orchestra.run.smithi079.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-16T14:04:12.349 INFO:teuthology.orchestra.run.smithi079.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-16T14:04:12.350 INFO:teuthology.orchestra.run.smithi079.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-16T14:04:12.356 INFO:teuthology.orchestra.run.smithi079.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-16T14:04:12.399 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-16T14:04:12.505 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-16T14:04:12.514 INFO:teuthology.orchestra.run.smithi079.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-16T14:04:12.514 INFO:teuthology.orchestra.run.smithi079.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-16T14:04:12.514 INFO:teuthology.orchestra.run.smithi079.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-16T14:04:12.517 INFO:teuthology.orchestra.run.smithi079.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-16T14:04:12.519 INFO:teuthology.orchestra.run.smithi079.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-16T14:04:12.561 INFO:teuthology.orchestra.run.smithi079.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-16T14:04:12.566 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-16T14:04:12.570 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-16T14:04:12.574 INFO:teuthology.orchestra.run.smithi079.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-16T14:04:12.578 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-16T14:04:12.622 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-16T14:04:12.635 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-16T14:04:12.643 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-16T14:04:12.644 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-16T14:04:12.649 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-16T14:04:12.650 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-16T14:04:12.653 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-16T14:04:12.701 INFO:teuthology.orchestra.run.smithi079.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-16T14:04:12.707 INFO:teuthology.orchestra.run.smithi079.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-16T14:04:12.736 INFO:teuthology.orchestra.run.smithi079.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-16T14:04:12.762 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-16T14:04:12.767 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-16T14:04:12.791 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-16T14:04:12.792 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-16T14:04:12.796 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-16T14:04:12.807 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-16T14:04:12.878 INFO:teuthology.orchestra.run.smithi079.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-16T14:04:12.886 INFO:teuthology.orchestra.run.smithi079.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-16T14:04:12.908 INFO:teuthology.orchestra.run.smithi079.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-16T14:04:12.915 INFO:teuthology.orchestra.run.smithi079.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-16T14:04:12.916 INFO:teuthology.orchestra.run.smithi079.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-16T14:04:12.923 INFO:teuthology.orchestra.run.smithi079.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-16T14:04:12.924 INFO:teuthology.orchestra.run.smithi079.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-16T14:04:12.925 INFO:teuthology.orchestra.run.smithi079.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-16T14:04:12.925 INFO:teuthology.orchestra.run.smithi079.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-16T14:04:12.937 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-16T14:04:12.946 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-16T14:04:12.960 INFO:teuthology.orchestra.run.smithi079.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-16T14:04:12.968 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-16T14:04:12.975 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-16T14:04:12.976 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-16T14:04:12.982 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-16T14:04:12.984 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-16T14:04:12.984 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-16T14:04:12.985 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-16T14:04:13.012 INFO:teuthology.orchestra.run.smithi079.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-16T14:04:13.013 INFO:teuthology.orchestra.run.smithi079.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-16T14:04:13.013 INFO:teuthology.orchestra.run.smithi079.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-16T14:04:13.014 INFO:teuthology.orchestra.run.smithi079.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-16T14:04:13.016 INFO:teuthology.orchestra.run.smithi079.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T14:04:13.017 INFO:teuthology.orchestra.run.smithi079.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-16T14:04:13.018 INFO:teuthology.orchestra.run.smithi079.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-16T14:04:13.020 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-16T14:04:13.020 INFO:teuthology.orchestra.run.smithi079.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-16T14:04:13.042 INFO:teuthology.orchestra.run.smithi079.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-16T14:04:13.073 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-16T14:04:13.074 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-16T14:04:13.074 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-16T14:04:13.075 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-16T14:04:13.077 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T14:04:13.078 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-16T14:04:13.078 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-16T14:04:13.081 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-16T14:04:13.101 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-16T14:04:13.102 INFO:teuthology.orchestra.run.smithi079.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-16T14:04:13.104 INFO:teuthology.orchestra.run.smithi079.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-16T14:04:13.104 INFO:teuthology.orchestra.run.smithi079.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-16T14:04:13.110 INFO:teuthology.orchestra.run.smithi079.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-16T14:04:13.111 INFO:teuthology.orchestra.run.smithi079.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-16T14:04:13.124 INFO:teuthology.orchestra.run.smithi079.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-16T14:04:13.125 INFO:teuthology.orchestra.run.smithi079.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-16T14:04:13.127 INFO:teuthology.orchestra.run.smithi079.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-16T14:04:13.132 INFO:teuthology.orchestra.run.smithi079.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-16T14:04:13.164 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-16T14:04:13.166 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-16T14:04:13.167 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-16T14:04:13.176 INFO:teuthology.orchestra.run.smithi164.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-16T14:04:13.176 INFO:teuthology.orchestra.run.smithi164.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-16T14:04:13.177 INFO:teuthology.orchestra.run.smithi164.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-16T14:04:13.178 INFO:teuthology.orchestra.run.smithi164.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-16T14:04:13.184 INFO:teuthology.orchestra.run.smithi164.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-16T14:04:13.186 INFO:teuthology.orchestra.run.smithi164.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-16T14:04:13.205 INFO:teuthology.orchestra.run.smithi079.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T14:04:13.206 INFO:teuthology.orchestra.run.smithi079.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-16T14:04:13.207 INFO:teuthology.orchestra.run.smithi079.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-16T14:04:13.208 INFO:teuthology.orchestra.run.smithi079.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-16T14:04:13.209 INFO:teuthology.orchestra.run.smithi079.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-16T14:04:13.265 INFO:teuthology.orchestra.run.smithi164.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T14:04:13.265 INFO:teuthology.orchestra.run.smithi164.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-16T14:04:13.269 INFO:teuthology.orchestra.run.smithi164.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-16T14:04:13.271 INFO:teuthology.orchestra.run.smithi164.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-16T14:04:13.273 INFO:teuthology.orchestra.run.smithi164.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-16T14:04:13.273 INFO:teuthology.orchestra.run.smithi164.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-16T14:04:13.275 INFO:teuthology.orchestra.run.smithi164.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-16T14:04:13.286 INFO:teuthology.orchestra.run.smithi079.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-16T14:04:13.287 INFO:teuthology.orchestra.run.smithi079.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-16T14:04:13.473 INFO:teuthology.orchestra.run.smithi079.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-16T14:04:13.542 INFO:teuthology.orchestra.run.smithi079.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-16T14:04:13.545 INFO:teuthology.orchestra.run.smithi079.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-16T14:04:13.548 INFO:teuthology.orchestra.run.smithi079.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-16T14:04:13.570 INFO:teuthology.orchestra.run.smithi079.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-16T14:04:13.597 INFO:teuthology.orchestra.run.smithi079.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-16T14:04:13.646 INFO:teuthology.orchestra.run.smithi079.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-16T14:04:13.647 INFO:teuthology.orchestra.run.smithi164.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-16T14:04:13.648 INFO:teuthology.orchestra.run.smithi079.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-16T14:04:13.668 INFO:teuthology.orchestra.run.smithi164.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-16T14:04:13.678 INFO:teuthology.orchestra.run.smithi164.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-16T14:04:13.721 INFO:teuthology.orchestra.run.smithi079.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-16T14:04:13.721 INFO:teuthology.orchestra.run.smithi079.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-16T14:04:13.721 INFO:teuthology.orchestra.run.smithi079.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-16T14:04:13.722 INFO:teuthology.orchestra.run.smithi079.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-16T14:04:13.722 INFO:teuthology.orchestra.run.smithi079.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-16T14:04:13.723 INFO:teuthology.orchestra.run.smithi079.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-16T14:04:13.723 INFO:teuthology.orchestra.run.smithi079.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-16T14:04:13.732 INFO:teuthology.orchestra.run.smithi079.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-16T14:04:13.780 INFO:teuthology.orchestra.run.smithi079.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-16T14:04:13.780 INFO:teuthology.orchestra.run.smithi079.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-16T14:04:13.802 INFO:teuthology.orchestra.run.smithi164.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-16T14:04:13.803 INFO:teuthology.orchestra.run.smithi164.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-16T14:04:13.850 INFO:teuthology.orchestra.run.smithi164.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-16T14:04:13.850 INFO:teuthology.orchestra.run.smithi164.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-16T14:04:13.851 INFO:teuthology.orchestra.run.smithi164.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-16T14:04:13.852 INFO:teuthology.orchestra.run.smithi164.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-16T14:04:13.853 INFO:teuthology.orchestra.run.smithi164.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-16T14:04:13.862 INFO:teuthology.orchestra.run.smithi079.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-16T14:04:13.865 INFO:teuthology.orchestra.run.smithi079.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-16T14:04:13.868 INFO:teuthology.orchestra.run.smithi079.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-16T14:04:13.870 INFO:teuthology.orchestra.run.smithi079.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-16T14:04:13.870 INFO:teuthology.orchestra.run.smithi079.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-16T14:04:13.873 INFO:teuthology.orchestra.run.smithi079.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-16T14:04:13.876 INFO:teuthology.orchestra.run.smithi079.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-16T14:04:13.892 INFO:teuthology.orchestra.run.smithi164.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-16T14:04:13.895 INFO:teuthology.orchestra.run.smithi164.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-16T14:04:13.895 INFO:teuthology.orchestra.run.smithi164.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-16T14:04:13.897 INFO:teuthology.orchestra.run.smithi164.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-16T14:04:13.902 INFO:teuthology.orchestra.run.smithi164.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-16T14:04:13.953 INFO:teuthology.orchestra.run.smithi079.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-16T14:04:13.956 INFO:teuthology.orchestra.run.smithi079.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-16T14:04:13.959 INFO:teuthology.orchestra.run.smithi164.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-16T14:04:13.962 INFO:teuthology.orchestra.run.smithi164.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-16T14:04:13.977 INFO:teuthology.orchestra.run.smithi164.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-16T14:04:13.980 INFO:teuthology.orchestra.run.smithi164.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-16T14:04:13.981 INFO:teuthology.orchestra.run.smithi164.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-16T14:04:13.983 INFO:teuthology.orchestra.run.smithi164.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-16T14:04:13.988 INFO:teuthology.orchestra.run.smithi164.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-16T14:04:13.991 INFO:teuthology.orchestra.run.smithi164.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-16T14:04:14.044 INFO:teuthology.orchestra.run.smithi164.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-16T14:04:14.061 INFO:teuthology.orchestra.run.smithi164.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-16T14:04:14.067 INFO:teuthology.orchestra.run.smithi164.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-16T14:04:14.120 INFO:teuthology.orchestra.run.smithi079.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-16T14:04:14.123 INFO:teuthology.orchestra.run.smithi079.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-16T14:04:14.124 INFO:teuthology.orchestra.run.smithi079.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-16T14:04:14.125 INFO:teuthology.orchestra.run.smithi079.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-16T14:04:14.126 INFO:teuthology.orchestra.run.smithi079.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-16T14:04:14.128 INFO:teuthology.orchestra.run.smithi079.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-16T14:04:14.144 INFO:teuthology.orchestra.run.smithi079.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T14:04:14.144 INFO:teuthology.orchestra.run.smithi079.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-16T14:04:14.173 INFO:teuthology.orchestra.run.smithi079.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-16T14:04:14.204 INFO:teuthology.orchestra.run.smithi164.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-16T14:04:14.205 INFO:teuthology.orchestra.run.smithi079.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-16T14:04:14.232 INFO:teuthology.orchestra.run.smithi164.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-16T14:04:14.234 INFO:teuthology.orchestra.run.smithi164.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-16T14:04:14.234 INFO:teuthology.orchestra.run.smithi164.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-16T14:04:14.235 INFO:teuthology.orchestra.run.smithi164.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-16T14:04:14.236 INFO:teuthology.orchestra.run.smithi164.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-16T14:04:14.238 INFO:teuthology.orchestra.run.smithi164.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-16T14:04:14.252 INFO:teuthology.orchestra.run.smithi164.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T14:04:14.252 INFO:teuthology.orchestra.run.smithi164.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-16T14:04:14.252 INFO:teuthology.orchestra.run.smithi164.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-16T14:04:14.287 INFO:teuthology.orchestra.run.smithi079.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-16T14:04:14.288 INFO:teuthology.orchestra.run.smithi079.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-16T14:04:14.289 INFO:teuthology.orchestra.run.smithi079.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-16T14:04:14.292 INFO:teuthology.orchestra.run.smithi079.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-16T14:04:14.293 INFO:teuthology.orchestra.run.smithi079.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-16T14:04:14.317 INFO:teuthology.orchestra.run.smithi164.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-16T14:04:14.399 INFO:teuthology.orchestra.run.smithi164.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-16T14:04:14.400 INFO:teuthology.orchestra.run.smithi164.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-16T14:04:14.400 INFO:teuthology.orchestra.run.smithi164.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-16T14:04:14.404 INFO:teuthology.orchestra.run.smithi164.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-16T14:04:14.406 INFO:teuthology.orchestra.run.smithi164.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-16T14:04:14.459 INFO:teuthology.orchestra.run.smithi079.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-16T14:04:14.459 INFO:teuthology.orchestra.run.smithi079.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-16T14:04:14.474 INFO:teuthology.orchestra.run.smithi079.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-16T14:04:14.491 INFO:teuthology.orchestra.run.smithi079.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-16T14:04:14.492 INFO:teuthology.orchestra.run.smithi079.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-16T14:04:14.507 INFO:teuthology.orchestra.run.smithi079.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-16T14:04:14.702 INFO:teuthology.orchestra.run.smithi079.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-16T14:04:14.914 INFO:teuthology.orchestra.run.smithi079.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-16T14:04:14.914 INFO:teuthology.orchestra.run.smithi079.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-16T14:04:15.143 INFO:teuthology.orchestra.run.smithi164.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-16T14:04:15.144 INFO:teuthology.orchestra.run.smithi164.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-16T14:04:15.156 INFO:teuthology.orchestra.run.smithi164.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-16T14:04:15.209 INFO:teuthology.orchestra.run.smithi164.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-16T14:04:15.210 INFO:teuthology.orchestra.run.smithi164.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-16T14:04:15.211 INFO:teuthology.orchestra.run.smithi164.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-16T14:04:15.519 INFO:teuthology.orchestra.run.smithi164.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-16T14:04:15.773 INFO:teuthology.orchestra.run.smithi164.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-16T14:04:15.774 INFO:teuthology.orchestra.run.smithi164.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-16T14:04:16.690 INFO:teuthology.orchestra.run.smithi079.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-16T14:04:16.694 INFO:teuthology.orchestra.run.smithi079.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-16T14:04:16.695 INFO:teuthology.orchestra.run.smithi079.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-16T14:04:17.191 INFO:teuthology.orchestra.run.smithi079.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-16T14:04:17.590 INFO:teuthology.orchestra.run.smithi164.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-16T14:04:17.594 INFO:teuthology.orchestra.run.smithi164.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-16T14:04:17.595 INFO:teuthology.orchestra.run.smithi164.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-16T14:04:17.845 INFO:teuthology.orchestra.run.smithi079.stdout:Fetched 201 MB in 5s (38.2 MB/s) 2024-09-16T14:04:17.908 INFO:teuthology.orchestra.run.smithi164.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-16T14:04:18.003 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-16T14:04:18.060 INFO:teuthology.orchestra.run.smithi079.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-16T14:04:18.064 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-16T14:04:18.156 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T14:04:18.509 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:18.549 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 201 MB in 6s (33.9 MB/s) 2024-09-16T14:04:18.635 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T14:04:18.719 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-16T14:04:18.774 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-16T14:04:18.779 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-16T14:04:18.868 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T14:04:19.187 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:19.305 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T14:04:19.364 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:19.482 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T14:04:20.028 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:20.151 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T14:04:20.169 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T14:04:20.185 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:20.236 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:20.572 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-rados. 2024-09-16T14:04:20.589 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:20.630 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:20.821 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T14:04:20.838 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:20.871 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:20.958 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T14:04:20.976 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:21.008 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:21.224 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rados. 2024-09-16T14:04:21.243 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:21.261 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T14:04:21.277 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:21.282 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:21.327 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:21.635 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T14:04:21.650 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:21.672 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T14:04:21.691 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:21.693 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:21.730 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:21.954 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T14:04:21.972 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:22.012 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:22.084 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T14:04:22.101 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-16T14:04:22.142 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-16T14:04:22.281 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T14:04:22.298 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:22.339 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:22.403 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T14:04:22.419 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:22.453 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:22.703 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T14:04:22.719 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-16T14:04:22.761 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-16T14:04:22.831 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-16T14:04:22.849 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-16T14:04:22.889 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T14:04:23.047 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T14:04:23.067 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:23.105 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:23.276 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-16T14:04:23.293 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-16T14:04:23.334 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T14:04:23.468 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-16T14:04:23.483 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-16T14:04:23.524 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T14:04:23.646 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T14:04:23.664 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-16T14:04:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T14:04:23.845 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-16T14:04:23.861 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-16T14:04:23.894 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T14:04:24.082 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T14:04:24.098 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-16T14:04:24.166 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T14:04:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T14:04:24.292 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-16T14:04:24.330 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T14:04:24.527 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T14:04:24.547 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-16T14:04:24.594 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T14:04:24.709 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T14:04:24.725 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-16T14:04:24.776 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T14:04:25.048 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T14:04:25.065 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T14:04:25.086 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T14:04:25.105 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-16T14:04:25.116 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:04:25.136 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T14:04:25.607 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T14:04:25.622 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T14:04:25.651 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T14:04:25.667 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T14:04:25.675 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:04:25.734 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:04:26.063 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T14:04:26.080 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T14:04:26.129 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:04:26.210 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T14:04:26.226 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T14:04:26.268 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:04:26.533 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-16T14:04:26.549 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-16T14:04:26.591 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T14:04:26.613 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T14:04:26.632 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T14:04:26.696 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:04:26.961 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package librgw2. 2024-09-16T14:04:26.977 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:27.019 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:27.074 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-16T14:04:27.090 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-16T14:04:27.132 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T14:04:27.519 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librgw2. 2024-09-16T14:04:27.535 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:27.577 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:27.830 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T14:04:27.848 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:27.880 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:28.233 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package liboath0. 2024-09-16T14:04:28.251 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-16T14:04:28.291 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-16T14:04:28.346 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T14:04:28.362 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:28.404 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:28.636 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T14:04:28.653 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:28.694 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:28.715 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liboath0. 2024-09-16T14:04:28.732 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-16T14:04:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-16T14:04:29.098 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-common. 2024-09-16T14:04:29.116 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:29.173 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:29.293 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T14:04:29.310 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:29.368 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:29.847 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-common. 2024-09-16T14:04:29.864 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:29.904 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:31.771 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-base. 2024-09-16T14:04:31.786 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:31.839 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:32.377 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-base. 2024-09-16T14:04:32.393 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:32.445 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-cryptography. 2024-09-16T14:04:32.758 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-16T14:04:32.800 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T14:04:33.188 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T14:04:33.207 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-16T14:04:33.271 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-16T14:04:33.361 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cryptography. 2024-09-16T14:04:33.380 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-16T14:04:33.423 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T14:04:33.549 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-packaging. 2024-09-16T14:04:33.565 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-16T14:04:33.624 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-16T14:04:33.810 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T14:04:33.826 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-16T14:04:33.868 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-16T14:04:33.876 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T14:04:33.893 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:33.934 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:34.221 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-packaging. 2024-09-16T14:04:34.240 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-16T14:04:34.279 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-16T14:04:34.324 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T14:04:34.343 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-16T14:04:34.407 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T14:04:34.532 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T14:04:34.548 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:34.607 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:34.726 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T14:04:34.746 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-16T14:04:34.801 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-16T14:04:34.995 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T14:04:35.014 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-16T14:04:35.053 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T14:04:35.122 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T14:04:35.140 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-16T14:04:35.197 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-16T14:04:35.390 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T14:04:35.406 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-16T14:04:35.448 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-16T14:04:35.492 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-jwt. 2024-09-16T14:04:35.511 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-16T14:04:35.567 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T14:04:35.810 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T14:04:35.827 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-16T14:04:35.868 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-16T14:04:35.879 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T14:04:35.894 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-16T14:04:35.962 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-16T14:04:36.179 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jwt. 2024-09-16T14:04:36.195 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-16T14:04:36.237 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T14:04:36.265 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T14:04:36.282 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-16T14:04:36.331 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-16T14:04:36.566 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T14:04:36.582 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-16T14:04:36.615 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-16T14:04:36.643 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-mako. 2024-09-16T14:04:36.658 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-16T14:04:36.726 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T14:04:36.919 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T14:04:36.937 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-16T14:04:36.977 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-16T14:04:37.088 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T14:04:37.107 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T14:04:37.146 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T14:04:37.296 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-mako. 2024-09-16T14:04:37.314 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-16T14:04:37.354 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T14:04:37.491 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T14:04:37.507 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-16T14:04:37.549 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T14:04:37.691 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T14:04:37.711 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T14:04:37.749 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T14:04:37.852 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-webob. 2024-09-16T14:04:37.871 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-16T14:04:37.911 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T14:04:38.019 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T14:04:38.037 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-16T14:04:38.069 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T14:04:38.256 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-16T14:04:38.272 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-16T14:04:38.314 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T14:04:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webob. 2024-09-16T14:04:38.421 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-16T14:04:38.462 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T14:04:38.642 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-bs4. 2024-09-16T14:04:38.660 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-16T14:04:38.700 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-16T14:04:38.790 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-16T14:04:38.806 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-16T14:04:38.865 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T14:04:39.080 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T14:04:39.097 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-16T14:04:39.142 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T14:04:39.176 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-bs4. 2024-09-16T14:04:39.193 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-16T14:04:39.234 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-16T14:04:39.500 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T14:04:39.517 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-16T14:04:39.558 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-16T14:04:39.629 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T14:04:39.646 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-16T14:04:39.690 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T14:04:39.895 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-paste. 2024-09-16T14:04:39.911 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-16T14:04:39.954 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T14:04:40.015 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T14:04:40.036 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-16T14:04:40.082 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-16T14:04:40.316 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T14:04:40.331 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-16T14:04:40.368 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-paste. 2024-09-16T14:04:40.374 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T14:04:40.388 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-16T14:04:40.435 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T14:04:40.710 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T14:04:40.726 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-16T14:04:40.768 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-16T14:04:40.780 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T14:04:40.796 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-16T14:04:40.838 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T14:04:41.115 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T14:04:41.125 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T14:04:41.132 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-16T14:04:41.144 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-16T14:04:41.173 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-16T14:04:41.182 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-16T14:04:41.510 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T14:04:41.523 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T14:04:41.526 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-16T14:04:41.543 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-16T14:04:41.568 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T14:04:41.581 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-16T14:04:41.909 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T14:04:41.925 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-16T14:04:41.942 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T14:04:41.960 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-16T14:04:41.967 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T14:04:42.002 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T14:04:42.322 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T14:04:42.338 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-16T14:04:42.372 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T14:04:42.379 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T14:04:42.389 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:42.463 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:42.766 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T14:04:42.782 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:42.824 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:42.850 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T14:04:42.868 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:42.908 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:43.202 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T14:04:43.218 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:43.269 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:43.435 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T14:04:43.450 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:43.493 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:43.781 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T14:04:43.798 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:43.839 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:44.330 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T14:04:44.346 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:44.405 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:45.385 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T14:04:45.402 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:45.443 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:46.874 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph. 2024-09-16T14:04:46.892 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:46.932 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:47.269 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T14:04:47.285 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:47.327 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:47.513 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph. 2024-09-16T14:04:47.529 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:47.571 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:47.789 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T14:04:47.808 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:47.847 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:47.874 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T14:04:47.892 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:47.932 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:48.343 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T14:04:48.359 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:48.400 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package cephadm. 2024-09-16T14:04:48.402 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:48.417 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:48.458 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:48.794 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T14:04:48.812 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-16T14:04:48.852 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T14:04:48.955 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephadm. 2024-09-16T14:04:48.971 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:04:49.013 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:49.197 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T14:04:49.213 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:49.255 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:49.333 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T14:04:49.352 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-16T14:04:49.390 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T14:04:49.625 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T14:04:49.641 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T14:04:49.683 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T14:04:49.727 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T14:04:49.745 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:49.785 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:49.960 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libjs-underscore. 2024-09-16T14:04:49.977 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-16T14:04:50.018 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T14:04:50.122 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T14:04:50.139 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T14:04:50.172 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T14:04:50.321 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-16T14:04:50.337 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-16T14:04:50.379 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T14:04:50.425 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjs-underscore. 2024-09-16T14:04:50.444 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-16T14:04:50.516 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T14:04:50.740 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-routes. 2024-09-16T14:04:50.757 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-16T14:04:50.768 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-16T14:04:50.784 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-16T14:04:50.798 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-16T14:04:50.826 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T14:04:51.152 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T14:04:51.169 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:51.188 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-routes. 2024-09-16T14:04:51.203 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-16T14:04:51.209 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:51.246 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-16T14:04:51.567 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T14:04:51.586 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:51.625 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:52.590 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-decorator. 2024-09-16T14:04:52.609 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-16T14:04:52.640 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T14:04:52.956 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-decorator. 2024-09-16T14:04:52.960 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-16T14:04:52.972 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-16T14:04:52.977 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-16T14:04:53.014 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T14:04:53.019 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T14:04:53.308 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-16T14:04:53.325 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-16T14:04:53.366 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T14:04:53.371 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-scipy. 2024-09-16T14:04:53.389 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-16T14:04:53.449 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-16T14:04:53.728 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-scipy. 2024-09-16T14:04:53.743 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-16T14:04:53.786 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-16T14:04:54.847 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-16T14:04:54.867 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-16T14:04:54.905 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T14:04:55.078 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-16T14:04:55.097 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-16T14:04:55.136 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T14:04:55.492 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T14:04:55.509 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-16T14:04:55.550 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-16T14:04:55.657 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T14:04:55.672 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-16T14:04:55.706 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-16T14:04:55.954 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T14:04:55.972 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-16T14:04:56.012 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T14:04:56.085 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T14:04:56.102 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-16T14:04:56.142 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T14:04:56.795 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T14:04:56.811 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:56.853 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:56.859 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T14:04:56.878 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:04:56.917 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:04:57.998 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T14:04:58.014 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-16T14:04:58.041 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T14:04:58.056 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-16T14:04:58.058 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-16T14:04:58.097 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-16T14:04:58.376 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T14:04:58.395 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-16T14:04:58.425 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T14:04:58.433 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T14:04:58.442 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-16T14:04:58.483 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T14:04:58.787 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T14:04:58.807 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-16T14:04:58.812 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T14:04:58.832 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-16T14:04:58.870 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T14:04:58.878 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T14:04:59.198 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T14:04:59.207 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T14:04:59.214 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-16T14:04:59.225 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-16T14:04:59.265 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-16T14:04:59.281 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-16T14:04:59.576 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T14:04:59.596 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-16T14:04:59.610 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T14:04:59.629 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-16T14:04:59.667 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-16T14:04:59.668 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-16T14:04:59.989 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-blinker. 2024-09-16T14:05:00.006 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-16T14:05:00.010 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-blinker. 2024-09-16T14:05:00.029 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-16T14:05:00.047 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T14:05:00.079 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T14:05:00.375 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-16T14:05:00.391 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-16T14:05:00.416 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-16T14:05:00.432 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T14:05:00.435 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-16T14:05:00.474 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T14:05:00.827 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T14:05:00.843 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-16T14:05:00.885 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T14:05:01.128 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T14:05:01.148 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-16T14:05:01.180 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T14:05:01.186 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T14:05:01.200 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-16T14:05:01.230 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T14:05:01.526 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T14:05:01.543 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-16T14:05:01.567 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T14:05:01.583 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-16T14:05:01.584 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T14:05:01.643 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T14:05:01.879 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T14:05:01.898 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-16T14:05:01.937 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T14:05:02.545 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T14:05:02.562 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:05:02.602 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:02.636 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T14:05:02.654 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:05:02.686 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:02.939 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T14:05:02.958 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-16T14:05:02.997 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-16T14:05:03.039 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T14:05:03.059 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-16T14:05:03.120 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-16T14:05:03.376 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T14:05:03.394 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T14:05:03.434 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T14:05:03.509 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T14:05:03.528 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T14:05:03.567 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T14:05:03.654 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package jq. 2024-09-16T14:05:03.670 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T14:05:03.703 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T14:05:03.836 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package jq. 2024-09-16T14:05:03.856 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T14:05:03.911 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T14:05:03.968 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package socat. 2024-09-16T14:05:03.988 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-16T14:05:04.026 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-16T14:05:04.172 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package socat. 2024-09-16T14:05:04.188 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-16T14:05:04.238 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-16T14:05:04.289 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T14:05:04.308 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-16T14:05:04.339 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-16T14:05:04.568 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T14:05:04.585 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-16T14:05:04.626 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-16T14:05:04.696 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-test. 2024-09-16T14:05:04.716 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:05:04.754 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:05.001 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-test. 2024-09-16T14:05:05.020 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:05:05.076 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:09.367 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T14:05:09.385 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:05:09.425 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:09.664 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T14:05:09.680 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T14:05:09.729 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-16T14:05:09.748 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:09.749 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T14:05:09.787 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T14:05:10.087 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-16T14:05:10.103 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T14:05:10.162 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T14:05:10.333 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T14:05:10.350 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:05:10.391 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:10.682 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T14:05:10.701 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:05:10.740 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:10.786 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T14:05:10.802 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-16T14:05:10.861 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T14:05:11.169 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T14:05:11.174 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-16T14:05:11.186 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-16T14:05:11.191 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T14:05:11.233 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T14:05:11.244 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T14:05:11.582 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-16T14:05:11.600 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T14:05:11.661 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T14:05:12.837 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-16T14:05:12.858 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-16T14:05:12.903 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-16T14:05:13.264 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-16T14:05:13.283 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-16T14:05:13.322 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T14:05:13.412 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-16T14:05:13.430 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-16T14:05:13.469 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-16T14:05:13.609 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-attr. 2024-09-16T14:05:13.629 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-16T14:05:13.666 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-16T14:05:13.839 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-16T14:05:13.855 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-16T14:05:13.897 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T14:05:14.003 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webencodings. 2024-09-16T14:05:14.022 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-16T14:05:14.061 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T14:05:14.233 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-attr. 2024-09-16T14:05:14.250 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-16T14:05:14.291 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-16T14:05:14.389 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-html5lib. 2024-09-16T14:05:14.407 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-16T14:05:14.447 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-16T14:05:14.619 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-webencodings. 2024-09-16T14:05:14.640 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-16T14:05:14.677 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T14:05:14.792 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-16T14:05:14.809 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-16T14:05:14.850 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T14:05:14.997 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-html5lib. 2024-09-16T14:05:15.014 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-16T14:05:15.055 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-16T14:05:15.312 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T14:05:15.328 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T14:05:15.361 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T14:05:15.383 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-16T14:05:15.403 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-16T14:05:15.441 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T14:05:15.690 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T14:05:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-16T14:05:15.747 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-16T14:05:15.953 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T14:05:15.973 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T14:05:16.011 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T14:05:16.068 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T14:05:16.084 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-16T14:05:16.146 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T14:05:16.373 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T14:05:16.393 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-16T14:05:16.430 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-16T14:05:16.479 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-py. 2024-09-16T14:05:16.498 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-16T14:05:16.537 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T14:05:16.767 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T14:05:16.784 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-16T14:05:16.825 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T14:05:16.882 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T14:05:16.899 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-16T14:05:16.940 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T14:05:17.178 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-py. 2024-09-16T14:05:17.195 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-16T14:05:17.236 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T14:05:17.243 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T14:05:17.262 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-16T14:05:17.301 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T14:05:17.589 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T14:05:17.609 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-16T14:05:17.640 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T14:05:17.647 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T14:05:17.657 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-16T14:05:17.698 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-16T14:05:18.000 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T14:05:18.018 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-16T14:05:18.058 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T14:05:18.068 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T14:05:18.087 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-16T14:05:18.126 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T14:05:18.396 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T14:05:18.402 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T14:05:18.413 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-16T14:05:18.419 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-16T14:05:18.454 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T14:05:18.460 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-16T14:05:18.822 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T14:05:18.839 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-16T14:05:18.881 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T14:05:19.183 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T14:05:19.204 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-16T14:05:19.241 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T14:05:19.317 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package radosgw. 2024-09-16T14:05:19.351 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:05:19.375 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:20.007 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package radosgw. 2024-09-16T14:05:20.026 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:05:20.064 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:21.170 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T14:05:21.187 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:05:21.228 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:21.538 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-16T14:05:21.827 INFO:teuthology.orchestra.run.smithi079.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T14:05:21.827 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T14:05:21.846 INFO:teuthology.orchestra.run.smithi079.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T14:05:21.876 INFO:teuthology.orchestra.run.smithi079.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:21.953 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T14:05:22.079 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T14:05:22.183 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-16T14:05:22.272 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:22.481 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T14:05:22.640 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T14:05:22.766 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T14:05:22.860 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user cephadm....done 2024-09-16T14:05:22.986 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:23.030 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T14:05:23.156 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T14:05:23.485 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T14:05:23.581 INFO:teuthology.orchestra.run.smithi079.stdout:Adding system user cephadm....done 2024-09-16T14:05:23.751 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T14:05:23.803 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T14:05:23.877 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T14:05:23.929 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-16T14:05:24.055 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T14:05:24.221 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T14:05:24.406 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-16T14:05:24.522 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T14:05:24.640 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-16T14:05:24.766 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T14:05:24.791 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T14:05:25.076 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-16T14:05:25.135 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-16T14:05:25.402 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T14:05:25.545 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T14:05:25.679 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T14:05:25.863 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-16T14:05:25.996 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:26.156 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T14:05:26.272 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T14:05:26.398 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-16T14:05:26.465 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T14:05:26.684 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T14:05:26.766 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:26.967 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T14:05:27.094 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T14:05:27.094 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T14:05:27.260 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-16T14:05:27.494 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-16T14:05:27.564 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T14:05:27.812 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T14:05:27.854 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T14:05:27.996 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T14:05:28.088 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T14:05:28.381 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-16T14:05:28.439 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T14:05:28.673 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T14:05:28.715 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T14:05:28.974 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T14:05:29.041 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-16T14:05:29.317 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-16T14:05:29.326 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T14:05:29.619 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T14:05:29.643 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T14:05:29.769 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T14:05:29.895 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T14:05:29.945 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-16T14:05:30.179 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T14:05:30.271 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-16T14:05:30.547 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T14:05:30.597 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T14:05:30.706 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T14:05:30.832 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T14:05:30.965 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T14:05:31.108 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T14:05:31.317 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-16T14:05:31.477 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:31.501 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T14:05:31.869 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T14:05:32.120 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:05:32.229 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-16T14:05:32.254 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-16T14:05:32.439 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:32.597 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T14:05:32.723 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-16T14:05:32.826 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-16T14:05:33.098 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:05:33.125 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T14:05:33.233 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-16T14:05:33.437 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:05:33.561 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T14:05:33.584 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T14:05:33.687 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T14:05:33.710 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-16T14:05:33.853 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-16T14:05:34.005 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T14:05:34.162 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T14:05:34.256 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-16T14:05:34.463 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:05:34.541 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T14:05:34.614 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T14:05:34.658 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T14:05:34.740 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T14:05:34.960 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-16T14:05:35.058 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T14:05:35.252 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T14:05:35.343 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-16T14:05:35.553 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T14:05:35.656 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T14:05:35.786 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T14:05:35.955 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-16T14:05:36.079 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-16T14:05:36.223 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-16T14:05:36.372 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T14:05:36.382 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T14:05:36.700 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-16T14:05:36.706 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T14:05:37.051 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T14:05:37.133 INFO:teuthology.orchestra.run.smithi079.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-16T14:05:37.177 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T14:05:37.303 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T14:05:37.384 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-16T14:05:37.510 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T14:05:37.629 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T14:05:37.844 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-16T14:05:37.905 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:38.023 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-16T14:05:38.195 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T14:05:38.321 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T14:05:38.472 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T14:05:38.790 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T14:05:39.075 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:39.226 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-16T14:05:39.470 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:39.596 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T14:05:39.714 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-16T14:05:40.073 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-16T14:05:40.358 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-16T14:05:40.651 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:05:40.706 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:40.785 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-16T14:05:40.832 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T14:05:40.967 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-16T14:05:41.119 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:41.270 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T14:05:41.368 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-16T14:05:41.589 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:41.694 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-16T14:05:41.715 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T14:05:41.970 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T14:05:42.058 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T14:05:42.096 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-16T14:05:42.464 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:42.615 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T14:05:42.975 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:43.088 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:43.101 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T14:05:43.214 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T14:05:43.340 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-16T14:05:43.452 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T14:05:43.465 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T14:05:43.575 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-16T14:05:43.901 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T14:05:44.027 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-16T14:05:44.312 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:44.438 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:44.524 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:44.650 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T14:05:44.666 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T14:05:44.666 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T14:05:44.801 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-16T14:05:44.952 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T14:05:45.078 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-16T14:05:45.122 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:45.325 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T14:05:45.379 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T14:05:45.505 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-16T14:05:45.619 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T14:05:45.797 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:45.923 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:45.945 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T14:05:46.172 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T14:05:46.173 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T14:05:46.297 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:46.423 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:46.550 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:46.608 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:46.692 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:46.826 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-16T14:05:46.844 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T14:05:47.128 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-16T14:05:47.137 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T14:05:47.479 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:47.480 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T14:05:47.605 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:47.731 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T14:05:47.867 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:48.026 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:48.152 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:48.286 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:48.437 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-16T14:05:48.643 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:48.747 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-16T14:05:49.032 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:49.191 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:49.212 INFO:teuthology.orchestra.run.smithi164.stdout:Adding group ceph....done 2024-09-16T14:05:49.317 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T14:05:49.563 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user ceph....done 2024-09-16T14:05:49.765 INFO:teuthology.orchestra.run.smithi164.stdout:Setting system user ceph properties....done 2024-09-16T14:05:49.796 INFO:teuthology.orchestra.run.smithi164.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T14:05:49.951 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T14:05:50.262 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:50.272 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T14:05:50.719 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:50.845 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:50.848 INFO:teuthology.orchestra.run.smithi079.stdout:Adding group ceph....done 2024-09-16T14:05:51.141 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:05:51.141 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:05:51.249 INFO:teuthology.orchestra.run.smithi079.stdout:Adding system user ceph....done 2024-09-16T14:05:51.424 INFO:teuthology.orchestra.run.smithi079.stdout:Setting system user ceph properties....done 2024-09-16T14:05:51.449 INFO:teuthology.orchestra.run.smithi079.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T14:05:51.592 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T14:05:51.789 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:51.910 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T14:05:52.164 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T14:05:52.355 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:52.480 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:52.599 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:52.788 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:05:52.788 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:05:52.816 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T14:05:52.816 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T14:05:53.251 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:53.433 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:53.483 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T14:05:53.483 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T14:05:53.811 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T14:05:53.918 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:54.268 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:54.449 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T14:05:54.449 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T14:05:54.486 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T14:05:54.486 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T14:05:54.879 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:54.919 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:55.004 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:55.138 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T14:05:55.138 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T14:05:55.147 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:55.400 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T14:05:55.400 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T14:05:55.587 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:55.840 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:55.974 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:56.083 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:56.115 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T14:05:56.115 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T14:05:56.226 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:56.345 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-16T14:05:56.505 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-16T14:05:56.572 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:56.706 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:56.756 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-16T14:05:56.874 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:57.091 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T14:05:57.091 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T14:05:57.548 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:57.693 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:57.818 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:57.969 INFO:teuthology.orchestra.run.smithi079.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T14:05:58.096 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-16T14:05:58.256 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-16T14:05:58.667 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-16T14:05:59.559 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-16T14:06:00.152 INFO:teuthology.orchestra.run.smithi164.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T14:06:00.158 DEBUG:teuthology.parallel:result is None 2024-09-16T14:06:01.511 INFO:teuthology.orchestra.run.smithi079.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-16T14:06:02.150 INFO:teuthology.orchestra.run.smithi079.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T14:06:02.155 DEBUG:teuthology.parallel:result is None 2024-09-16T14:06:02.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T14:06:02.304 DEBUG:teuthology.orchestra.run.smithi079:> dpkg-query -W -f '${Version}' ceph 2024-09-16T14:06:02.327 INFO:teuthology.orchestra.run.smithi079.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-16T14:06:02.327 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-16T14:06:02.327 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-16T14:06:02.329 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T14:06:02.468 DEBUG:teuthology.orchestra.run.smithi164:> dpkg-query -W -f '${Version}' ceph 2024-09-16T14:06:02.492 INFO:teuthology.orchestra.run.smithi164.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-16T14:06:02.492 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-16T14:06:02.492 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-16T14:06:02.494 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T14:06:02.494 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:02.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T14:06:02.511 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:02.511 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T14:06:02.550 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T14:06:02.550 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:02.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T14:06:02.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T14:06:02.581 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:02.582 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T14:06:02.597 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T14:06:02.606 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T14:06:02.606 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:02.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T14:06:02.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T14:06:02.680 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:02.680 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T14:06:02.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T14:06:02.757 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T14:06:02.758 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:02.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T14:06:02.774 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T14:06:02.784 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:02.784 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T14:06:02.800 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T14:06:02.856 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T14:06:02.930 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T14:06:02.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /var/log/ceph 2024-09-16T14:06:02.934 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-09-16T14:06:02.949 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T14:06:02.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T14:06:02.952 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T14:06:02.961 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T14:06:02.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T14:06:02.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T14:06:03.028 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T14:06:03.028 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T14:06:03.028 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read}', 'email': 'sangadi@redhat.com', 'first_in_suite': False, 'job_id': '7907316', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_sangadi@teuthology', 'priority': 130, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9764, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi079.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_10:44:09', 'tube': 'smithi', 'user': 'sangadi', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T14:06:03.028 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T14:06:03.031 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T14:06:03.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T14:06:03.040 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T14:06:03.054 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:03.054 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:06:03.104 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T14:06:03.104 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-09-16T14:06:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T14:06:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:06:03.111 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 369 Links: 1 2024-09-16T14:06:03.111 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:06:03.111 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-16 14:02:36.940990695 +0000 2024-09-16T14:06:03.111 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-16 14:02:36.748995122 +0000 2024-09-16T14:06:03.111 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-16 14:02:36.748995122 +0000 2024-09-16T14:06:03.111 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-09-16T14:06:03.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T14:06:03.124 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-16T14:06:03.124 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-16T14:06:03.125 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000433727 s, 1.2 MB/s 2024-09-16T14:06:03.126 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T14:06:03.134 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-09-16T14:06:03.141 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T14:06:03.141 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:06:03.141 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-09-16T14:06:03.141 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:06:03.141 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-16 14:02:37.168985437 +0000 2024-09-16T14:06:03.141 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-16 14:02:37.164985529 +0000 2024-09-16T14:06:03.141 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-16 14:02:37.164985529 +0000 2024-09-16T14:06:03.141 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-09-16T14:06:03.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T14:06:03.156 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-16T14:06:03.156 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-16T14:06:03.156 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000363152 s, 1.4 MB/s 2024-09-16T14:06:03.157 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T14:06:03.206 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-09-16T14:06:03.213 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T14:06:03.213 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:06:03.213 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-09-16T14:06:03.213 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:06:03.213 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-16 14:02:37.612975198 +0000 2024-09-16T14:06:03.213 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-16 14:02:37.608975290 +0000 2024-09-16T14:06:03.213 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-16 14:02:37.608975290 +0000 2024-09-16T14:06:03.213 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-09-16T14:06:03.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T14:06:03.225 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-16T14:06:03.225 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-16T14:06:03.225 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000428278 s, 1.2 MB/s 2024-09-16T14:06:03.226 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T14:06:03.278 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-09-16T14:06:03.285 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T14:06:03.285 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:06:03.285 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 387 Links: 1 2024-09-16T14:06:03.285 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:06:03.285 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-09-16 14:02:38.236960807 +0000 2024-09-16T14:06:03.285 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-09-16 14:02:38.040965328 +0000 2024-09-16T14:06:03.285 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-09-16 14:02:38.040965328 +0000 2024-09-16T14:06:03.285 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2024-09-16T14:06:03.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T14:06:03.299 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-09-16T14:06:03.299 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-09-16T14:06:03.300 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000367829 s, 1.4 MB/s 2024-09-16T14:06:03.300 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T14:06:03.350 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:03.350 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:03.350 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:06:03.357 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T14:06:03.357 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-09-16T14:06:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T14:06:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:06:03.364 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-09-16T14:06:03.364 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:06:03.364 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 14:02:34.376203913 +0000 2024-09-16T14:06:03.364 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 14:02:34.164208804 +0000 2024-09-16T14:06:03.364 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 14:02:34.164208804 +0000 2024-09-16T14:06:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-09-16T14:06:03.365 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T14:06:03.380 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T14:06:03.380 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T14:06:03.380 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000430555 s, 1.2 MB/s 2024-09-16T14:06:03.381 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T14:06:03.388 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-09-16T14:06:03.395 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T14:06:03.395 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:06:03.395 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-16T14:06:03.395 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:06:03.395 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 14:02:34.780194594 +0000 2024-09-16T14:06:03.395 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 14:02:34.588199023 +0000 2024-09-16T14:06:03.395 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 14:02:34.588199023 +0000 2024-09-16T14:06:03.395 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-09-16T14:06:03.395 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T14:06:03.406 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T14:06:03.406 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T14:06:03.406 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.00041746 s, 1.2 MB/s 2024-09-16T14:06:03.407 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T14:06:03.414 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-09-16T14:06:03.465 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T14:06:03.465 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:06:03.465 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 381 Links: 1 2024-09-16T14:06:03.465 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:06:03.466 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 14:02:35.196184999 +0000 2024-09-16T14:06:03.466 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 14:02:35.000189519 +0000 2024-09-16T14:06:03.466 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 14:02:35.000189519 +0000 2024-09-16T14:06:03.466 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-09-16T14:06:03.466 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T14:06:03.480 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T14:06:03.481 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T14:06:03.481 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000397365 s, 1.3 MB/s 2024-09-16T14:06:03.481 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T14:06:03.489 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-09-16T14:06:03.537 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T14:06:03.537 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:06:03.537 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 389 Links: 1 2024-09-16T14:06:03.538 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:06:03.538 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 14:02:35.612175402 +0000 2024-09-16T14:06:03.538 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 14:02:35.420179831 +0000 2024-09-16T14:06:03.538 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 14:02:35.420179831 +0000 2024-09-16T14:06:03.538 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-09-16T14:06:03.538 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T14:06:03.549 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T14:06:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T14:06:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000360512 s, 1.4 MB/s 2024-09-16T14:06:03.551 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T14:06:03.559 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:03.559 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T14:06:03.559 INFO:tasks.ceph:Generating config... 2024-09-16T14:06:03.560 INFO:tasks.ceph:[client] rbd cache = True 2024-09-16T14:06:03.560 INFO:tasks.ceph:[client] rbd clone copy on read = True 2024-09-16T14:06:03.560 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T14:06:03.560 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-16T14:06:03.560 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-16T14:06:03.560 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T14:06:03.561 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T14:06:03.561 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T14:06:03.561 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T14:06:03.561 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T14:06:03.561 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T14:06:03.562 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T14:06:03.562 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T14:06:03.562 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T14:06:03.562 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T14:06:03.562 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T14:06:03.562 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-16T14:06:03.562 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T14:06:03.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T14:06:03.598 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T14:06:03.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T14:06:03.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T14:06:03.674 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.79'), ('mon.c', '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]'), ('mon.b', '172.21.15.164')] 2024-09-16T14:06:03.674 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.164', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd clone copy on read': True}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-16T14:06:03.675 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:03.675 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T14:06:03.679 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.79 --addv c '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]' --add b 172.21.15.164 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: generated fsid a22b38a6-94db-470d-bf35-45d7605f3bb0 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:setting min_mon_release = octopus 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 0 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:fsid a22b38a6-94db-470d-bf35-45d7605f3bb0 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:last_changed 2024-09-16T14:06:03.758644+0000 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:created 2024-09-16T14:06:03.758644+0000 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:min_mon_release 15 (octopus) 2024-09-16T14:06:03.759 INFO:teuthology.orchestra.run.smithi079.stdout:election_strategy: 1 2024-09-16T14:06:03.760 INFO:teuthology.orchestra.run.smithi079.stdout:0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-09-16T14:06:03.760 INFO:teuthology.orchestra.run.smithi079.stdout:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-16T14:06:03.760 INFO:teuthology.orchestra.run.smithi079.stdout:2: [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] mon.c 2024-09-16T14:06:03.760 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T14:06:03.761 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T14:06:03.767 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a22b38a6-94db-470d-bf35-45d7605f3bb0... 2024-09-16T14:06:03.767 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T14:06:03.770 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: chdir = "" 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: auth supported = cephx 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: filestore xattr use omap = true 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: mon clock drift allowed = 1.000 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.798 INFO:teuthology.orchestra.run.smithi079.stdout: osd crush chooseleaf type = 0 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: auth debug = true 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on old message = true 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on bug = true 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: mon pg warn max object skew = 0 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default size = 2 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow primary affinity = true 2024-09-16T14:06:03.799 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow pg remap = true 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on legacy crush tunables = false 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on crush straw calc version zero = false 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on no sortbitwise = false 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on osd down out interval zero = false 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on too few osds = false 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: mon_allow_pool_size_one = true 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: osd default data pool replay window = 5 2024-09-16T14:06:03.800 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: mon allow pool delete = true 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: mon cluster log file level = debug 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: debug asserts on shutdown = true 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: mon health detail to clog = false 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: mon host = "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.164" 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: mon client directed command retry = 5 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on pool no app = False 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: ms inject socket failures = 5000 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = a22b38a6-94db-470d-bf35-45d7605f3bb0 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout:[osd] 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: osd journal size = 100 2024-09-16T14:06:03.801 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub load threshold = 5.0 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub max interval = 600 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd mclock profile = high_recovery_ops 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd recover clone overlap = true 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd recovery max chunk = 1048576 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug shutdown = true 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug op order = true 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify stray on activate = true 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd open classes on start = true 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug pg log writeout = true 2024-09-16T14:06:03.802 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: osd deep scrub update digest min age = 30 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: osd map max advance = 10 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: journal zero on create = true 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: filestore ondisk finisher threads = 3 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: filestore apply finisher threads = 3 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: bdev debug aio = true 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug misdirected ops = true 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore block size = 96636764160 2024-09-16T14:06:03.803 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore compression algorithm = zstd 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore compression mode = aggressive 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore fsck on mount = True 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluefs = 1/20 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluestore = 1/20 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: debug osd = 20 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: debug rocksdb = 4/10 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd full ratio = 0.9 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd nearfull ratio = 0.8 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: osd failsafe full ratio = 0.95 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: osd max markdown count = 1000 2024-09-16T14:06:03.804 INFO:teuthology.orchestra.run.smithi079.stdout: osd objectstore = bluestore 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: osd shutdown pgref assert = True 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr] 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: debug mgr = 20 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: mgr/telemetry/nag = false 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout:[mon] 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-09-16T14:06:03.805 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: debug paxos = 20 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: mon data avail warn = 5 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: mon mgr mkfs grace = 240 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd reporter subtree level = osd 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd prime pg temp = true 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T14:06:03.806 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: # 1m isn't quite enough 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: mon_down_mkfs_grace = 2m 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_filestore_osds = false 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout:[client] 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: rgw cache enabled = true 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable ops log = true 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable usage log = true 2024-09-16T14:06:03.807 INFO:teuthology.orchestra.run.smithi079.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T14:06:03.808 INFO:teuthology.orchestra.run.smithi079.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T14:06:03.808 INFO:teuthology.orchestra.run.smithi079.stdout: rbd cache = True 2024-09-16T14:06:03.808 INFO:teuthology.orchestra.run.smithi079.stdout: rbd clone copy on read = True 2024-09-16T14:06:03.808 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.a] 2024-09-16T14:06:03.808 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.c] 2024-09-16T14:06:03.808 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.b] 2024-09-16T14:06:03.808 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-09-16T14:06:03.809 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-09-16T14:06:03.810 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-09-16T14:06:03.811 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.164" 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: mon client directed command retry = 5 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on pool no app = False 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: ms inject socket failures = 5000 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = a22b38a6-94db-470d-bf35-45d7605f3bb0 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-09-16T14:06:03.812 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.813 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression algorithm = zstd 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression mode = aggressive 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 1/20 2024-09-16T14:06:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 1/20 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 4/10 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: osd max markdown count = 1000 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: osd shutdown pgref assert = True 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-09-16T14:06:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:06:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-09-16T14:06:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: rbd cache = True 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: rbd clone copy on read = True 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-09-16T14:06:03.818 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-09-16T14:06:03.820 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T14:06:03.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T14:06:03.866 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T14:06:03.866 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:03.866 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T14:06:03.916 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:03.916 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T14:06:03.923 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:06:03.923 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:03.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T14:06:03.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T14:06:03.946 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:03.946 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:03.996 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:06:03.996 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:03.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T14:06:03.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T14:06:04.023 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:04.023 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:04.073 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T14:06:04.073 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T14:06:04.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T14:06:04.122 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T14:06:04.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T14:06:04.167 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T14:06:04.170 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T14:06:04.170 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T14:06:04.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T14:06:04.206 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T14:06:04.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T14:06:04.255 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T14:06:04.268 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T14:06:04.268 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T14:06:04.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T14:06:04.282 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:04.282 INFO:tasks.ceph:role: osd.0 2024-09-16T14:06:04.282 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:06:04.282 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T14:06:04.790 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:06:04.791 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:06:04.793 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-16T14:06:04.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T14:06:04.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T14:06:04.892 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T14:06:04.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:06:04.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T14:06:04.904 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:04.904 INFO:tasks.ceph:role: osd.1 2024-09-16T14:06:04.904 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:06:04.904 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T14:06:05.425 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:06:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:06:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:06:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 2024-09-16T14:06:05.425 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:06:05.425 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:06:05.426 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:06:05.426 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T14:06:05.426 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:06:05.426 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:06:05.427 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-16T14:06:05.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T14:06:05.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T14:06:05.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:06:05.500 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T14:06:05.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T14:06:05.557 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:05.557 INFO:tasks.ceph:role: osd.2 2024-09-16T14:06:05.557 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:06:05.557 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T14:06:06.021 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:06:06.022 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:06:06.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-16T14:06:06.024 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T14:06:06.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T14:06:06.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:06:06.104 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T14:06:06.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T14:06:06.115 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:06.115 INFO:tasks.ceph:role: osd.3 2024-09-16T14:06:06.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:06:06.115 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T14:06:06.601 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:06:06.601 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:06:06.601 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:06:06.601 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 2024-09-16T14:06:06.601 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:06:06.601 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:06:06.601 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:06:06.601 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T14:06:06.602 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:06:06.602 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:06:06.603 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-09-16T14:06:06.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T14:06:06.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T14:06:06.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:06:06.676 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T14:06:06.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:06.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:06.748+0000 7fca726bf540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T14:06:06.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:06.748+0000 7fca726bf540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T14:06:06.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:06.748+0000 7fca726bf540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T14:06:06.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:06.748+0000 7fca726bf540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T14:06:06.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:06.748+0000 7fca726bf540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T14:06:06.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:06.752+0000 7fca726bf540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T14:06:07.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:07.308+0000 7fca726bf540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:08.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:08.092+0000 7fca726bf540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:08.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:08.872+0000 7fca726bf540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:09.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T14:06:09.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:09.776+0000 7f852e80d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T14:06:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:09.776+0000 7f852e80d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T14:06:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:09.776+0000 7f852e80d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T14:06:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:09.776+0000 7f852e80d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T14:06:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:09.776+0000 7f852e80d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T14:06:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:09.776+0000 7f852e80d540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T14:06:10.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:10.328+0000 7f852e80d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:11.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:11.120+0000 7f852e80d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:11.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:11.900+0000 7f852e80d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:12.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T14:06:12.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:12.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:12.868+0000 7f0f2693c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T14:06:12.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:12.868+0000 7f0f2693c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T14:06:12.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:12.868+0000 7f0f2693c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T14:06:12.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:12.868+0000 7f0f2693c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T14:06:12.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:12.868+0000 7f0f2693c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T14:06:12.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:12.868+0000 7f0f2693c540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T14:06:13.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:13.420+0000 7f0f2693c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:14.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:14.204+0000 7f0f2693c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:14.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:14.984+0000 7f0f2693c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:15.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T14:06:15.846 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:15.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:15.880+0000 7f87ba85a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T14:06:15.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:15.880+0000 7f87ba85a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T14:06:15.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:15.880+0000 7f87ba85a540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T14:06:15.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:15.880+0000 7f87ba85a540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T14:06:15.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:15.880+0000 7f87ba85a540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T14:06:15.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:15.880+0000 7f87ba85a540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T14:06:16.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:16.436+0000 7f87ba85a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:17.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:17.220+0000 7f87ba85a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:18.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:06:18.000+0000 7f87ba85a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:18.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T14:06:18.870 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T14:06:18.886 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:18.887 INFO:tasks.ceph:role: osd.4 2024-09-16T14:06:18.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:06:18.887 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T14:06:19.419 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:06:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:06:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:06:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 2024-09-16T14:06:19.419 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:06:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:06:19.420 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:06:19.420 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T14:06:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:06:19.420 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:06:19.421 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-16T14:06:19.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T14:06:19.498 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T14:06:19.511 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T14:06:19.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:06:19.512 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T14:06:19.528 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:19.528 INFO:tasks.ceph:role: osd.5 2024-09-16T14:06:19.528 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:06:19.528 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:06:20.047 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:06:20.049 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-16T14:06:20.049 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T14:06:20.111 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T14:06:20.122 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T14:06:20.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:06:20.123 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T14:06:20.182 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:20.182 INFO:tasks.ceph:role: osd.6 2024-09-16T14:06:20.182 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:06:20.183 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:06:20.657 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:06:20.660 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-16T14:06:20.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T14:06:20.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T14:06:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T14:06:20.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:06:20.735 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T14:06:20.751 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:06:20.751 INFO:tasks.ceph:role: osd.7 2024-09-16T14:06:20.751 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:06:20.751 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T14:06:21.246 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:06:21.247 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:06:21.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-16T14:06:21.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T14:06:21.298 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T14:06:21.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:06:21.310 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T14:06:21.310 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:21.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:21.428+0000 7f03c11fc540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T14:06:21.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:21.428+0000 7f03c11fc540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T14:06:21.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:21.428+0000 7f03c11fc540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T14:06:21.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:21.428+0000 7f03c11fc540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T14:06:21.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:21.428+0000 7f03c11fc540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T14:06:21.442 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:21.436+0000 7f03c11fc540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T14:06:22.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:22.004+0000 7f03c11fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:22.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:22.792+0000 7f03c11fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:23.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:23.572+0000 7f03c11fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:24.427 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T14:06:24.444 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:24.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:24.476+0000 7f2b87435540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T14:06:24.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:24.480+0000 7f2b87435540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T14:06:24.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:24.480+0000 7f2b87435540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T14:06:24.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:24.480+0000 7f2b87435540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T14:06:24.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:24.480+0000 7f2b87435540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T14:06:24.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:24.480+0000 7f2b87435540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T14:06:25.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:25.040+0000 7f2b87435540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:25.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:25.832+0000 7f2b87435540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:26.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:26.620+0000 7f2b87435540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:27.479 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T14:06:27.539 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:27.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:27.568+0000 7fe0b8205540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T14:06:27.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:27.572+0000 7fe0b8205540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T14:06:27.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:27.572+0000 7fe0b8205540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T14:06:27.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:27.572+0000 7fe0b8205540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T14:06:27.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:27.572+0000 7fe0b8205540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T14:06:27.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:27.572+0000 7fe0b8205540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T14:06:28.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:28.140+0000 7fe0b8205540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:28.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:28.923+0000 7fe0b8205540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:29.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:29.707+0000 7fe0b8205540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:30.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T14:06:30.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:30.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:30.623+0000 7fc5e8ee3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T14:06:30.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:30.623+0000 7fc5e8ee3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T14:06:30.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:30.623+0000 7fc5e8ee3540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T14:06:30.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:30.623+0000 7fc5e8ee3540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T14:06:30.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:30.623+0000 7fc5e8ee3540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T14:06:30.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:30.623+0000 7fc5e8ee3540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T14:06:31.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:31.175+0000 7fc5e8ee3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:31.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:31.959+0000 7fc5e8ee3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:32.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:32.739+0000 7fc5e8ee3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T14:06:33.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T14:06:33.651 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T14:06:33.651 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:33.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T14:06:33.668 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:33.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T14:06:33.684 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:33.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T14:06:33.743 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:33.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T14:06:33.757 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:33.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T14:06:33.770 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:33.771 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T14:06:33.786 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:33.786 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T14:06:33.797 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:33.798 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T14:06:33.854 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:33.854 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T14:06:33.864 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:33.864 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T14:06:33.879 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:33.880 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T14:06:33.888 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:33.888 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T14:06:33.895 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T14:06:33.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T14:06:33.898 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T14:06:33.941 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.y] 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMO+hmM2xDBxAAUWtGvaPXsVHiMJuzHS3doA== 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.0] 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBOO+hmkxjBLBAA0DiM8u3ssabgJl3bmpSPQg== 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.1] 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBRO+hm1XdZLhAA6EgHPEgxwc92b14ma421sA== 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.2] 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBUO+hmxs7aMxAA8mxl5c+dNMP8g8UBBSvkug== 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.3] 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBXO+hmeHmiNBAAZZ1d56961ZKGnWcqP8hj7g== 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMO+hmZ4jrCRAA/IAAejqOCIFELIrxa3JfkQ== 2024-09-16T14:06:33.942 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBdO+hmsaylGRAA27+qxqBuau/NNZvsSvAjeg== 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBgO+hm9O+yHBAAqzirlVfgYCkh12Ir+moLtQ== 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.6] 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBjO+hm2EcxIhAApjfoGFxjql3iUEh5QO3EHw== 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.7] 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBmO+hmE4dbJRAAOCsAHsJiOL6uBb6Y8M1sNg== 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMO+hmyP9FDBAAQc27XQdhOdYHtRaQLBKSFg== 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout:[client.1] 2024-09-16T14:06:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMO+hmOEwiDxAA59aNAUSqmQIWbugk/G6wNg== 2024-09-16T14:06:33.951 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2024-09-16T14:06:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBMO+hmM2xDBxAAUWtGvaPXsVHiMJuzHS3doA== 2024-09-16T14:06:33.951 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBOO+hmkxjBLBAA0DiM8u3ssabgJl3bmpSPQg== 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBRO+hm1XdZLhAA6EgHPEgxwc92b14ma421sA== 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBUO+hmxs7aMxAA8mxl5c+dNMP8g8UBBSvkug== 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBXO+hmeHmiNBAAZZ1d56961ZKGnWcqP8hj7g== 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBMO+hmZ4jrCRAA/IAAejqOCIFELIrxa3JfkQ== 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBdO+hmsaylGRAA27+qxqBuau/NNZvsSvAjeg== 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2024-09-16T14:06:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBgO+hm9O+yHBAAqzirlVfgYCkh12Ir+moLtQ== 2024-09-16T14:06:33.953 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2024-09-16T14:06:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBjO+hm2EcxIhAApjfoGFxjql3iUEh5QO3EHw== 2024-09-16T14:06:33.953 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2024-09-16T14:06:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBmO+hmE4dbJRAAOCsAHsJiOL6uBb6Y8M1sNg== 2024-09-16T14:06:33.953 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-09-16T14:06:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBMO+hmyP9FDBAAQc27XQdhOdYHtRaQLBKSFg== 2024-09-16T14:06:33.953 INFO:teuthology.orchestra.run.smithi164.stdout:[client.1] 2024-09-16T14:06:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQBMO+hmOEwiDxAA59aNAUSqmQIWbugk/G6wNg== 2024-09-16T14:06:33.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T14:06:33.987 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T14:06:34.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.033 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.079 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.165 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.255 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T14:06:34.303 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T14:06:34.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.392 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.438 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.569 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:06:34.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T14:06:34.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T14:06:34.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T14:06:34.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T14:06:34.890 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T14:06:34.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T14:06:34.906 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T14:06:34.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T14:06:34.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T14:06:35.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T14:06:35.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T14:06:35.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T14:06:35.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T14:06:35.209 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T14:06:35.223 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:35.226 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:06:35.231 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T14:06:35.231 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T14:06:35.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T14:06:35.234 INFO:tasks.ceph.mon.a:Started 2024-09-16T14:06:35.234 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T14:06:35.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T14:06:35.236 INFO:tasks.ceph.mon.c:Started 2024-09-16T14:06:35.236 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T14:06:35.236 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T14:06:35.277 INFO:tasks.ceph.mon.b:Started 2024-09-16T14:06:35.277 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T14:06:35.278 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T14:06:35.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T14:06:35.280 INFO:tasks.ceph.mgr.y:Started 2024-09-16T14:06:35.280 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T14:06:35.280 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T14:06:35.282 INFO:tasks.ceph.mgr.x:Started 2024-09-16T14:06:35.282 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T14:06:35.282 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T14:06:40.686 INFO:teuthology.orchestra.run.smithi079.stderr:adjusted tunables profile to default 2024-09-16T14:06:40.696 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T14:06:40.696 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:40.696 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T14:06:40.706 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:40.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T14:06:40.722 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:40.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T14:06:40.733 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-09-16T14:06:40.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T14:06:40.745 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:40.745 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T14:06:40.760 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:40.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T14:06:40.794 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:40.791+0000 7faa90185000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:06:40.806 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:40.799+0000 7f89ce4b8000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:06:40.817 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:40.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T14:06:40.825 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T14:06:40.826 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T14:06:40.835 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new a060b161-fd5a-4cfb-bac8-f2acc32d7955 0 2024-09-16T14:06:41.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T14:06:41.187+0000 7fc3d7fff700 0 --2- 172.21.15.164:0/3113897604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc3e006f3e0 0x7fc3e013c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T14:06:41.290 INFO:teuthology.orchestra.run.smithi164.stdout:0 2024-09-16T14:06:41.303 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new f5ba4b39-ceb8-46b3-938e-57249ed84b87 1 2024-09-16T14:06:41.507 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-09-16T14:06:41.519 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 8b657f92-66c9-46a3-a31f-0bd0be414e4e 2 2024-09-16T14:06:41.524 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:41.519+0000 7faa90185000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:06:41.528 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:41.523+0000 7f89ce4b8000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:06:41.581 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:41.575+0000 7faa90185000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:06:41.586 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:41.579+0000 7f89ce4b8000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:06:41.696 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:41.691+0000 7faa90185000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:06:41.704 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:41.699+0000 7f89ce4b8000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:06:41.726 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-09-16T14:06:41.739 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 15937bd7-8964-4140-9f86-b3093c318d1e 3 2024-09-16T14:06:41.912 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:41.907+0000 7faa90185000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:06:41.924 INFO:teuthology.orchestra.run.smithi164.stdout:3 2024-09-16T14:06:41.937 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 714baf80-ade0-4e88-ad4e-5171add1fc6a 4 2024-09-16T14:06:41.942 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:41.935+0000 7f89ce4b8000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:06:41.960 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:41.955+0000 7faa90185000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:06:41.994 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:41.987+0000 7f89ce4b8000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:06:42.126 INFO:teuthology.orchestra.run.smithi164.stdout:4 2024-09-16T14:06:42.132 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:42.127+0000 7faa90185000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:06:42.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new e5a21b84-81ab-45da-a081-3b504f5c4f3d 5 2024-09-16T14:06:42.187 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:42.179+0000 7f89ce4b8000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:06:42.191 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:42.187+0000 7faa90185000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:06:42.251 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:42.247+0000 7faa90185000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:06:42.256 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:42.251+0000 7f89ce4b8000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:06:42.321 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:42.315+0000 7f89ce4b8000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:06:42.340 INFO:teuthology.orchestra.run.smithi164.stdout:5 2024-09-16T14:06:42.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new a009857f-f798-4154-9ff4-f524f60b6c61 6 2024-09-16T14:06:42.540 INFO:teuthology.orchestra.run.smithi164.stdout:6 2024-09-16T14:06:42.557 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new e3506002-ba8b-4251-8ef1-136334574a6d 7 2024-09-16T14:06:42.633 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:42.627+0000 7faa90185000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:06:42.727 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:42.719+0000 7f89ce4b8000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:06:42.745 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:42.739+0000 7faa90185000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:06:42.753 INFO:teuthology.orchestra.run.smithi164.stdout:7 2024-09-16T14:06:42.766 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T14:06:42.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T14:06:42.768 INFO:tasks.ceph.osd.0:Started 2024-09-16T14:06:42.768 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T14:06:42.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T14:06:42.770 INFO:tasks.ceph.osd.1:Started 2024-09-16T14:06:42.770 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T14:06:42.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T14:06:42.772 INFO:tasks.ceph.osd.2:Started 2024-09-16T14:06:42.773 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T14:06:42.773 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T14:06:42.775 INFO:tasks.ceph.osd.3:Started 2024-09-16T14:06:42.775 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T14:06:42.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T14:06:42.777 INFO:tasks.ceph.osd.4:Started 2024-09-16T14:06:42.777 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T14:06:42.778 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T14:06:42.779 INFO:tasks.ceph.osd.5:Started 2024-09-16T14:06:42.779 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T14:06:42.780 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T14:06:42.781 INFO:tasks.ceph.osd.6:Started 2024-09-16T14:06:42.782 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T14:06:42.782 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T14:06:42.784 INFO:tasks.ceph.osd.7:Started 2024-09-16T14:06:42.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:06:42.866 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:42.859+0000 7f89ce4b8000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:06:42.980 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:06:42.980 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":10,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:06:42.748450+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T14:06:42.748450+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:06:42.994 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T14:06:42.994 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T14:06:43.057 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.051+0000 7faa90185000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:06:43.129 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:06:43.123+0000 7f0a1fab3540 -1 Falling back to public interface 2024-09-16T14:06:43.132 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:06:43.127+0000 7f61457d0540 -1 Falling back to public interface 2024-09-16T14:06:43.139 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:06:43.135+0000 7feffd2de540 -1 Falling back to public interface 2024-09-16T14:06:43.152 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:06:43.147+0000 7fc4f2dbc540 -1 Falling back to public interface 2024-09-16T14:06:43.155 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:06:43.151+0000 7fa3ec40e540 -1 Falling back to public interface 2024-09-16T14:06:43.158 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:43.151+0000 7f89ce4b8000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:06:43.171 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:06:43.167+0000 7f169a030540 -1 Falling back to public interface 2024-09-16T14:06:43.174 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:06:43.171+0000 7fe54afbc540 -1 Falling back to public interface 2024-09-16T14:06:43.217 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.211+0000 7faa90185000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:06:43.287 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.283+0000 7faa90185000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:06:43.322 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:43.315+0000 7f89ce4b8000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:06:43.338 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.335+0000 7faa90185000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:06:43.388 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:43.383+0000 7f89ce4b8000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:06:43.394 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.391+0000 7faa90185000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:06:43.421 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:06:43.415+0000 7f08d4191540 -1 Falling back to public interface 2024-09-16T14:06:43.440 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:43.435+0000 7f89ce4b8000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:06:43.496 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:43.491+0000 7f89ce4b8000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:06:43.561 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.555+0000 7faa90185000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:06:43.663 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:43.659+0000 7f89ce4b8000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:06:43.724 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.719+0000 7faa90185000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:06:43.790 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.783+0000 7faa90185000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:06:43.798 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T14:06:43.831 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:43.823+0000 7f89ce4b8000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:06:43.897 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:43.891+0000 7f89ce4b8000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:06:43.947 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:43.943+0000 7faa90185000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:06:43.967 INFO:teuthology.misc.health.smithi079.stdout: 2024-09-16T14:06:43.967 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":10,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:06:42.748450+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T14:06:42.748450+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:06:43.977 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T14:06:44.021 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:44.015+0000 7faa90185000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:06:44.054 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:44.047+0000 7f89ce4b8000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:06:44.117 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:44.111+0000 7f89ce4b8000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:06:44.724 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:44.719+0000 7faa90185000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:06:44.817 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T14:06:44.811+0000 7f89ce4b8000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:06:45.773 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:45.767+0000 7faa791f9700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T14:06:45.773 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-09-16T14:06:45.767+0000 7faa791f9700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T14:06:46.712 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:06:46.707+0000 7f61457d0540 -1 osd.4 0 log_to_monitors true 2024-09-16T14:06:46.898 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:06:46.891+0000 7f0a1fab3540 -1 osd.5 0 log_to_monitors true 2024-09-16T14:06:46.958 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:06:46.955+0000 7feffd2de540 -1 osd.3 0 log_to_monitors true 2024-09-16T14:06:46.984 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:06:46.979+0000 7fa3ec40e540 -1 osd.1 0 log_to_monitors true 2024-09-16T14:06:47.036 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:06:47.031+0000 7fc4f2dbc540 -1 osd.7 0 log_to_monitors true 2024-09-16T14:06:47.036 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:06:47.031+0000 7fe54afbc540 -1 osd.2 0 log_to_monitors true 2024-09-16T14:06:47.059 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:06:47.055+0000 7f169a030540 -1 osd.0 0 log_to_monitors true 2024-09-16T14:06:47.245 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:06:47.239+0000 7f08d4191540 -1 osd.6 0 log_to_monitors true 2024-09-16T14:06:49.170 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:06:49.163+0000 7f6140f70700 -1 osd.4 0 waiting for initial osdmap 2024-09-16T14:06:49.177 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:06:49.171+0000 7f6139d13700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:06:50.241 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:06:50.235+0000 7f0a1b253700 -1 osd.5 0 waiting for initial osdmap 2024-09-16T14:06:50.248 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:06:50.243+0000 7f0a13ff6700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:06:50.258 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:06:50.251+0000 7fc4ee55c700 -1 osd.7 0 waiting for initial osdmap 2024-09-16T14:06:50.263 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:06:50.259+0000 7fc4e72ff700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:06:50.270 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:06:50.263+0000 7f08cf931700 -1 osd.6 0 waiting for initial osdmap 2024-09-16T14:06:50.275 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:06:50.271+0000 7f08c86d4700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:06:50.287 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:06:50.283+0000 7f16957d0700 -1 osd.0 0 waiting for initial osdmap 2024-09-16T14:06:50.292 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:06:50.287+0000 7fe54675c700 -1 osd.2 0 waiting for initial osdmap 2024-09-16T14:06:50.294 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:06:50.291+0000 7f168e573700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:06:50.297 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:06:50.291+0000 7fe53f4ff700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:06:50.297 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:06:50.291+0000 7fa3e7bae700 -1 osd.1 0 waiting for initial osdmap 2024-09-16T14:06:50.303 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:06:50.299+0000 7fa3e0951700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:06:50.309 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:06:50.303+0000 7feffa2a3700 -1 osd.3 0 waiting for initial osdmap 2024-09-16T14:06:50.315 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:06:50.311+0000 7feff1821700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:06:50.783 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T14:06:51.073 INFO:teuthology.misc.health.smithi079.stdout: 2024-09-16T14:06:51.074 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":14,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:06:50.175793+0000","last_up_change":"2024-09-16T14:06:50.175793+0000","last_in_change":"2024-09-16T14:06:42.748450+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:06:51.090 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-16T14:06:57.897 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T14:06:58.186 INFO:teuthology.misc.health.smithi079.stdout: 2024-09-16T14:06:58.186 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":18,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:06:54.878676+0000","last_up_change":"2024-09-16T14:06:51.260361+0000","last_in_change":"2024-09-16T14:06:42.748450+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":22196},{"type":"v1","addr":"172.21.15.79:6817","nonce":22196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":22196},{"type":"v1","addr":"172.21.15.79:6819","nonce":22196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":22196},{"type":"v1","addr":"172.21.15.79:6824","nonce":22196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":22196},{"type":"v1","addr":"172.21.15.79:6821","nonce":22196}]},"public_addr":"172.21.15.79:6817/22196","cluster_addr":"172.21.15.79:6819/22196","heartbeat_back_addr":"172.21.15.79:6824/22196","heartbeat_front_addr":"172.21.15.79:6821/22196","state":["exists","up"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":22193},{"type":"v1","addr":"172.21.15.79:6801","nonce":22193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":22193},{"type":"v1","addr":"172.21.15.79:6803","nonce":22193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":22193},{"type":"v1","addr":"172.21.15.79:6807","nonce":22193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":22193},{"type":"v1","addr":"172.21.15.79:6805","nonce":22193}]},"public_addr":"172.21.15.79:6801/22193","cluster_addr":"172.21.15.79:6803/22193","heartbeat_back_addr":"172.21.15.79:6807/22193","heartbeat_front_addr":"172.21.15.79:6805/22193","state":["exists","up"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:06:58.198 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T14:06:58.198 INFO:tasks.ceph:Creating RBD pool 2024-09-16T14:06:58.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T14:06:59.472 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'rbd' created 2024-09-16T14:06:59.483 DEBUG:teuthology.orchestra.run.smithi079:> rbd --cluster ceph pool init rbd 2024-09-16T14:07:02.510 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T14:07:02.511 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T14:07:02.511 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T14:07:02.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T14:07:02.512 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T14:07:02.851 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:02.866 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":22105},{"type":"v1","addr":"172.21.15.79:6833","nonce":22105}]},"active_addr":"172.21.15.79:6833/22105","active_change":"2024-09-16T14:06:44.744643+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3577399234}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":1459479709}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":2644074130}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3090391806}]}]} 2024-09-16T14:07:02.869 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T14:07:02.870 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T14:07:02.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:07:03.153 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:03.153 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":22,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:07:02.482445+0000","last_up_change":"2024-09-16T14:06:51.260361+0000","last_in_change":"2024-09-16T14:06:42.748450+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:06:58.526935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":22196},{"type":"v1","addr":"172.21.15.79:6817","nonce":22196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":22196},{"type":"v1","addr":"172.21.15.79:6819","nonce":22196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":22196},{"type":"v1","addr":"172.21.15.79:6824","nonce":22196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":22196},{"type":"v1","addr":"172.21.15.79:6821","nonce":22196}]},"public_addr":"172.21.15.79:6817/22196","cluster_addr":"172.21.15.79:6819/22196","heartbeat_back_addr":"172.21.15.79:6824/22196","heartbeat_front_addr":"172.21.15.79:6821/22196","state":["exists","up"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":22193},{"type":"v1","addr":"172.21.15.79:6801","nonce":22193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":22193},{"type":"v1","addr":"172.21.15.79:6803","nonce":22193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":22193},{"type":"v1","addr":"172.21.15.79:6807","nonce":22193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":22193},{"type":"v1","addr":"172.21.15.79:6805","nonce":22193}]},"public_addr":"172.21.15.79:6801/22193","cluster_addr":"172.21.15.79:6803/22193","heartbeat_back_addr":"172.21.15.79:6807/22193","heartbeat_front_addr":"172.21.15.79:6805/22193","state":["exists","up"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:07:03.167 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T14:07:03.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:07:03.455 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:03.455 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":22,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:07:02.482445+0000","last_up_change":"2024-09-16T14:06:51.260361+0000","last_in_change":"2024-09-16T14:06:42.748450+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:06:58.526935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":22196},{"type":"v1","addr":"172.21.15.79:6817","nonce":22196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":22196},{"type":"v1","addr":"172.21.15.79:6819","nonce":22196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":22196},{"type":"v1","addr":"172.21.15.79:6824","nonce":22196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":22196},{"type":"v1","addr":"172.21.15.79:6821","nonce":22196}]},"public_addr":"172.21.15.79:6817/22196","cluster_addr":"172.21.15.79:6819/22196","heartbeat_back_addr":"172.21.15.79:6824/22196","heartbeat_front_addr":"172.21.15.79:6821/22196","state":["exists","up"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":22193},{"type":"v1","addr":"172.21.15.79:6801","nonce":22193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":22193},{"type":"v1","addr":"172.21.15.79:6803","nonce":22193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":22193},{"type":"v1","addr":"172.21.15.79:6807","nonce":22193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":22193},{"type":"v1","addr":"172.21.15.79:6805","nonce":22193}]},"public_addr":"172.21.15.79:6801/22193","cluster_addr":"172.21.15.79:6803/22193","heartbeat_back_addr":"172.21.15.79:6807/22193","heartbeat_front_addr":"172.21.15.79:6805/22193","state":["exists","up"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:07:03.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T14:07:03.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T14:07:03.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T14:07:03.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T14:07:03.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T14:07:03.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T14:07:03.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T14:07:03.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T14:07:03.677 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:03.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T14:07:03.679 INFO:teuthology.orchestra.run.smithi079.stdout:60129542150 2024-09-16T14:07:03.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T14:07:03.686 INFO:teuthology.orchestra.run.smithi079.stdout:64424509447 2024-09-16T14:07:03.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T14:07:03.694 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2024-09-16T14:07:03.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T14:07:03.698 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:03.698 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T14:07:03.703 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:03.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T14:07:03.706 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:03.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T14:07:03.897 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:03.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T14:07:04.105 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2024-09-16T14:07:04.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-09-16T14:07:04.132 INFO:teuthology.orchestra.run.smithi079.stdout:60129542149 2024-09-16T14:07:04.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2024-09-16T14:07:04.164 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:04.168 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2024-09-16T14:07:04.168 INFO:teuthology.orchestra.run.smithi079.stdout:64424509444 2024-09-16T14:07:04.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.0 2024-09-16T14:07:04.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2024-09-16T14:07:04.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-09-16T14:07:04.199 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2024-09-16T14:07:04.210 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2024-09-16T14:07:04.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2024-09-16T14:07:04.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-16T14:07:04.293 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2024-09-16T14:07:04.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-09-16T14:07:05.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T14:07:05.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T14:07:05.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T14:07:05.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T14:07:05.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T14:07:05.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T14:07:05.227 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T14:07:05.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T14:07:05.573 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:05.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-09-16T14:07:05.592 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:05.613 INFO:teuthology.orchestra.run.smithi079.stdout:60129542150 2024-09-16T14:07:05.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-09-16T14:07:05.629 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:05.636 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2024-09-16T14:07:05.646 INFO:teuthology.orchestra.run.smithi079.stdout:64424509447 2024-09-16T14:07:05.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.0 2024-09-16T14:07:05.653 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:05.653 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:05.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-09-16T14:07:05.658 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:05.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.6 2024-09-16T14:07:05.668 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:05.688 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:05.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-09-16T14:07:05.704 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:05.707 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:05.713 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2024-09-16T14:07:05.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-16T14:07:05.726 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:05.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-09-16T14:07:05.726 DEBUG:teuthology.parallel:result is None 2024-09-16T14:07:05.726 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T14:07:05.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:07:05.992 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:05.995 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:07:06.005 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-16T14:07:04.763353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2389012,"kb_used_data":1824,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752585708,"statfs":{"total":773094113280,"available":770647764992,"internally_reserved":0,"allocated":1867776,"data_stored":1760542,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.300671"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503840+0000","last_change":"2024-09-16T14:07:02.504261+0000","last_active":"2024-09-16T14:07:02.503840+0000","last_peered":"2024-09-16T14:07:02.503840+0000","last_clean":"2024-09-16T14:07:02.503840+0000","last_became_active":"2024-09-16T14:07:00.475877+0000","last_became_peered":"2024-09-16T14:07:00.475877+0000","last_unstale":"2024-09-16T14:07:02.503840+0000","last_undegraded":"2024-09-16T14:07:02.503840+0000","last_fullsized":"2024-09-16T14:07:02.503840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:15.110271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020129200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.516454+0000","last_change":"2024-09-16T14:07:02.517047+0000","last_active":"2024-09-16T14:07:02.516454+0000","last_peered":"2024-09-16T14:07:02.516454+0000","last_clean":"2024-09-16T14:07:02.516454+0000","last_became_active":"2024-09-16T14:07:00.472006+0000","last_became_peered":"2024-09-16T14:07:00.472006+0000","last_unstale":"2024-09-16T14:07:02.516454+0000","last_undegraded":"2024-09-16T14:07:02.516454+0000","last_fullsized":"2024-09-16T14:07:02.516454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:02.291214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082399199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.502363+0000","last_change":"2024-09-16T14:07:02.502729+0000","last_active":"2024-09-16T14:07:02.502363+0000","last_peered":"2024-09-16T14:07:02.502363+0000","last_clean":"2024-09-16T14:07:02.502363+0000","last_became_active":"2024-09-16T14:07:00.476599+0000","last_became_peered":"2024-09-16T14:07:00.476599+0000","last_unstale":"2024-09-16T14:07:02.502363+0000","last_undegraded":"2024-09-16T14:07:02.502363+0000","last_fullsized":"2024-09-16T14:07:02.502363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:38.332613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056022899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.493750+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:07:02.493750+0000","last_peered":"2024-09-16T14:07:02.493750+0000","last_clean":"2024-09-16T14:07:02.493750+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:07:02.493750+0000","last_undegraded":"2024-09-16T14:07:02.493750+0000","last_fullsized":"2024-09-16T14:07:02.493750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503836+0000","last_change":"2024-09-16T14:07:02.503836+0000","last_active":"2024-09-16T14:07:02.503836+0000","last_peered":"2024-09-16T14:07:02.503836+0000","last_clean":"2024-09-16T14:07:02.503836+0000","last_became_active":"2024-09-16T14:07:00.572358+0000","last_became_peered":"2024-09-16T14:07:00.572358+0000","last_unstale":"2024-09-16T14:07:02.503836+0000","last_undegraded":"2024-09-16T14:07:02.503836+0000","last_fullsized":"2024-09-16T14:07:02.503836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:28.110370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673887,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T14:07:00.477791+0000","last_change":"2024-09-16T14:07:00.475860+0000","last_active":"2024-09-16T14:07:00.477791+0000","last_peered":"2024-09-16T14:07:00.477791+0000","last_clean":"2024-09-16T14:07:00.477791+0000","last_became_active":"2024-09-16T14:07:00.475232+0000","last_became_peered":"2024-09-16T14:07:00.475232+0000","last_unstale":"2024-09-16T14:07:00.477791+0000","last_undegraded":"2024-09-16T14:07:00.477791+0000","last_fullsized":"2024-09-16T14:07:00.477791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.457456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.502476+0000","last_change":"2024-09-16T14:07:02.502802+0000","last_active":"2024-09-16T14:07:02.502476+0000","last_peered":"2024-09-16T14:07:02.502476+0000","last_clean":"2024-09-16T14:07:02.502476+0000","last_became_active":"2024-09-16T14:07:00.480643+0000","last_became_peered":"2024-09-16T14:07:00.480643+0000","last_unstale":"2024-09-16T14:07:02.502476+0000","last_undegraded":"2024-09-16T14:07:02.502476+0000","last_fullsized":"2024-09-16T14:07:02.502476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:52.536849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503387+0000","last_change":"2024-09-16T14:07:02.503755+0000","last_active":"2024-09-16T14:07:02.503387+0000","last_peered":"2024-09-16T14:07:02.503387+0000","last_clean":"2024-09-16T14:07:02.503387+0000","last_became_active":"2024-09-16T14:07:00.569748+0000","last_became_peered":"2024-09-16T14:07:00.569748+0000","last_unstale":"2024-09-16T14:07:02.503387+0000","last_undegraded":"2024-09-16T14:07:02.503387+0000","last_fullsized":"2024-09-16T14:07:02.503387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:11.927535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015626559999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T14:07:01.486533+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:07:01.486533+0000","last_peered":"2024-09-16T14:07:01.486533+0000","last_clean":"2024-09-16T14:07:01.486533+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:07:01.486533+0000","last_undegraded":"2024-09-16T14:07:01.486533+0000","last_fullsized":"2024-09-16T14:07:01.486533+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":278528,"data_stored":664294,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":278528,"data_stored":664294,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":221184,"data_stored":73926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6859999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:07:06.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:07:06.280 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:06.283 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:07:06.293 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-16T14:07:04.763353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2389012,"kb_used_data":1824,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752585708,"statfs":{"total":773094113280,"available":770647764992,"internally_reserved":0,"allocated":1867776,"data_stored":1760542,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.300671"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503840+0000","last_change":"2024-09-16T14:07:02.504261+0000","last_active":"2024-09-16T14:07:02.503840+0000","last_peered":"2024-09-16T14:07:02.503840+0000","last_clean":"2024-09-16T14:07:02.503840+0000","last_became_active":"2024-09-16T14:07:00.475877+0000","last_became_peered":"2024-09-16T14:07:00.475877+0000","last_unstale":"2024-09-16T14:07:02.503840+0000","last_undegraded":"2024-09-16T14:07:02.503840+0000","last_fullsized":"2024-09-16T14:07:02.503840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:15.110271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020129200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.516454+0000","last_change":"2024-09-16T14:07:02.517047+0000","last_active":"2024-09-16T14:07:02.516454+0000","last_peered":"2024-09-16T14:07:02.516454+0000","last_clean":"2024-09-16T14:07:02.516454+0000","last_became_active":"2024-09-16T14:07:00.472006+0000","last_became_peered":"2024-09-16T14:07:00.472006+0000","last_unstale":"2024-09-16T14:07:02.516454+0000","last_undegraded":"2024-09-16T14:07:02.516454+0000","last_fullsized":"2024-09-16T14:07:02.516454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:02.291214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082399199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.502363+0000","last_change":"2024-09-16T14:07:02.502729+0000","last_active":"2024-09-16T14:07:02.502363+0000","last_peered":"2024-09-16T14:07:02.502363+0000","last_clean":"2024-09-16T14:07:02.502363+0000","last_became_active":"2024-09-16T14:07:00.476599+0000","last_became_peered":"2024-09-16T14:07:00.476599+0000","last_unstale":"2024-09-16T14:07:02.502363+0000","last_undegraded":"2024-09-16T14:07:02.502363+0000","last_fullsized":"2024-09-16T14:07:02.502363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:38.332613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056022899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.493750+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:07:02.493750+0000","last_peered":"2024-09-16T14:07:02.493750+0000","last_clean":"2024-09-16T14:07:02.493750+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:07:02.493750+0000","last_undegraded":"2024-09-16T14:07:02.493750+0000","last_fullsized":"2024-09-16T14:07:02.493750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503836+0000","last_change":"2024-09-16T14:07:02.503836+0000","last_active":"2024-09-16T14:07:02.503836+0000","last_peered":"2024-09-16T14:07:02.503836+0000","last_clean":"2024-09-16T14:07:02.503836+0000","last_became_active":"2024-09-16T14:07:00.572358+0000","last_became_peered":"2024-09-16T14:07:00.572358+0000","last_unstale":"2024-09-16T14:07:02.503836+0000","last_undegraded":"2024-09-16T14:07:02.503836+0000","last_fullsized":"2024-09-16T14:07:02.503836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:28.110370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673887,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T14:07:00.477791+0000","last_change":"2024-09-16T14:07:00.475860+0000","last_active":"2024-09-16T14:07:00.477791+0000","last_peered":"2024-09-16T14:07:00.477791+0000","last_clean":"2024-09-16T14:07:00.477791+0000","last_became_active":"2024-09-16T14:07:00.475232+0000","last_became_peered":"2024-09-16T14:07:00.475232+0000","last_unstale":"2024-09-16T14:07:00.477791+0000","last_undegraded":"2024-09-16T14:07:00.477791+0000","last_fullsized":"2024-09-16T14:07:00.477791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.457456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.502476+0000","last_change":"2024-09-16T14:07:02.502802+0000","last_active":"2024-09-16T14:07:02.502476+0000","last_peered":"2024-09-16T14:07:02.502476+0000","last_clean":"2024-09-16T14:07:02.502476+0000","last_became_active":"2024-09-16T14:07:00.480643+0000","last_became_peered":"2024-09-16T14:07:00.480643+0000","last_unstale":"2024-09-16T14:07:02.502476+0000","last_undegraded":"2024-09-16T14:07:02.502476+0000","last_fullsized":"2024-09-16T14:07:02.502476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:52.536849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503387+0000","last_change":"2024-09-16T14:07:02.503755+0000","last_active":"2024-09-16T14:07:02.503387+0000","last_peered":"2024-09-16T14:07:02.503387+0000","last_clean":"2024-09-16T14:07:02.503387+0000","last_became_active":"2024-09-16T14:07:00.569748+0000","last_became_peered":"2024-09-16T14:07:00.569748+0000","last_unstale":"2024-09-16T14:07:02.503387+0000","last_undegraded":"2024-09-16T14:07:02.503387+0000","last_fullsized":"2024-09-16T14:07:02.503387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:11.927535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015626559999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T14:07:01.486533+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:07:01.486533+0000","last_peered":"2024-09-16T14:07:01.486533+0000","last_clean":"2024-09-16T14:07:01.486533+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:07:01.486533+0000","last_undegraded":"2024-09-16T14:07:01.486533+0000","last_fullsized":"2024-09-16T14:07:01.486533+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":278528,"data_stored":664294,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":278528,"data_stored":664294,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":221184,"data_stored":73926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6859999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:07:06.294 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T14:07:06.294 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T14:07:06.294 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T14:07:06.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T14:07:06.595 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:06.595 INFO:teuthology.orchestra.run.smithi079.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T14:07:06.608 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T14:07:06.608 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T14:07:06.619 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T14:07:06.619 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T14:07:06.620 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T14:07:06.620 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T14:07:06.933 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T14:07:06.933 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T14:07:06.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:07:07.222 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:07.225 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:07:07.236 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T14:07:06.763935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2389012,"kb_used_data":1824,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752585708,"statfs":{"total":773094113280,"available":770647764992,"internally_reserved":0,"allocated":1867776,"data_stored":1760542,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.298495"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503840+0000","last_change":"2024-09-16T14:07:02.504261+0000","last_active":"2024-09-16T14:07:02.503840+0000","last_peered":"2024-09-16T14:07:02.503840+0000","last_clean":"2024-09-16T14:07:02.503840+0000","last_became_active":"2024-09-16T14:07:00.475877+0000","last_became_peered":"2024-09-16T14:07:00.475877+0000","last_unstale":"2024-09-16T14:07:02.503840+0000","last_undegraded":"2024-09-16T14:07:02.503840+0000","last_fullsized":"2024-09-16T14:07:02.503840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:15.110271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020129200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.516454+0000","last_change":"2024-09-16T14:07:02.517047+0000","last_active":"2024-09-16T14:07:02.516454+0000","last_peered":"2024-09-16T14:07:02.516454+0000","last_clean":"2024-09-16T14:07:02.516454+0000","last_became_active":"2024-09-16T14:07:00.472006+0000","last_became_peered":"2024-09-16T14:07:00.472006+0000","last_unstale":"2024-09-16T14:07:02.516454+0000","last_undegraded":"2024-09-16T14:07:02.516454+0000","last_fullsized":"2024-09-16T14:07:02.516454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:02.291214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082399199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.502363+0000","last_change":"2024-09-16T14:07:02.502729+0000","last_active":"2024-09-16T14:07:02.502363+0000","last_peered":"2024-09-16T14:07:02.502363+0000","last_clean":"2024-09-16T14:07:02.502363+0000","last_became_active":"2024-09-16T14:07:00.476599+0000","last_became_peered":"2024-09-16T14:07:00.476599+0000","last_unstale":"2024-09-16T14:07:02.502363+0000","last_undegraded":"2024-09-16T14:07:02.502363+0000","last_fullsized":"2024-09-16T14:07:02.502363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:38.332613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056022899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.493750+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:07:02.493750+0000","last_peered":"2024-09-16T14:07:02.493750+0000","last_clean":"2024-09-16T14:07:02.493750+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:07:02.493750+0000","last_undegraded":"2024-09-16T14:07:02.493750+0000","last_fullsized":"2024-09-16T14:07:02.493750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503836+0000","last_change":"2024-09-16T14:07:02.503836+0000","last_active":"2024-09-16T14:07:02.503836+0000","last_peered":"2024-09-16T14:07:02.503836+0000","last_clean":"2024-09-16T14:07:02.503836+0000","last_became_active":"2024-09-16T14:07:00.572358+0000","last_became_peered":"2024-09-16T14:07:00.572358+0000","last_unstale":"2024-09-16T14:07:02.503836+0000","last_undegraded":"2024-09-16T14:07:02.503836+0000","last_fullsized":"2024-09-16T14:07:02.503836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:28.110370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673887,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T14:07:00.477791+0000","last_change":"2024-09-16T14:07:00.475860+0000","last_active":"2024-09-16T14:07:00.477791+0000","last_peered":"2024-09-16T14:07:00.477791+0000","last_clean":"2024-09-16T14:07:00.477791+0000","last_became_active":"2024-09-16T14:07:00.475232+0000","last_became_peered":"2024-09-16T14:07:00.475232+0000","last_unstale":"2024-09-16T14:07:00.477791+0000","last_undegraded":"2024-09-16T14:07:00.477791+0000","last_fullsized":"2024-09-16T14:07:00.477791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.457456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.502476+0000","last_change":"2024-09-16T14:07:02.502802+0000","last_active":"2024-09-16T14:07:02.502476+0000","last_peered":"2024-09-16T14:07:02.502476+0000","last_clean":"2024-09-16T14:07:02.502476+0000","last_became_active":"2024-09-16T14:07:00.480643+0000","last_became_peered":"2024-09-16T14:07:00.480643+0000","last_unstale":"2024-09-16T14:07:02.502476+0000","last_undegraded":"2024-09-16T14:07:02.502476+0000","last_fullsized":"2024-09-16T14:07:02.502476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:52.536849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503387+0000","last_change":"2024-09-16T14:07:02.503755+0000","last_active":"2024-09-16T14:07:02.503387+0000","last_peered":"2024-09-16T14:07:02.503387+0000","last_clean":"2024-09-16T14:07:02.503387+0000","last_became_active":"2024-09-16T14:07:00.569748+0000","last_became_peered":"2024-09-16T14:07:00.569748+0000","last_unstale":"2024-09-16T14:07:02.503387+0000","last_undegraded":"2024-09-16T14:07:02.503387+0000","last_fullsized":"2024-09-16T14:07:02.503387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:11.927535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015626559999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T14:07:01.486533+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:07:01.486533+0000","last_peered":"2024-09-16T14:07:01.486533+0000","last_clean":"2024-09-16T14:07:01.486533+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:07:01.486533+0000","last_undegraded":"2024-09-16T14:07:01.486533+0000","last_fullsized":"2024-09-16T14:07:01.486533+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":278528,"data_stored":664294,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":278528,"data_stored":664294,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":221184,"data_stored":73926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:07:07.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:07:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:07.532 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:07:07.543 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T14:07:06.763935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2389012,"kb_used_data":1824,"kb_used_omap":13,"kb_used_meta":2387058,"kb_avail":752585708,"statfs":{"total":773094113280,"available":770647764992,"internally_reserved":0,"allocated":1867776,"data_stored":1760542,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2444348096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.298495"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503840+0000","last_change":"2024-09-16T14:07:02.504261+0000","last_active":"2024-09-16T14:07:02.503840+0000","last_peered":"2024-09-16T14:07:02.503840+0000","last_clean":"2024-09-16T14:07:02.503840+0000","last_became_active":"2024-09-16T14:07:00.475877+0000","last_became_peered":"2024-09-16T14:07:00.475877+0000","last_unstale":"2024-09-16T14:07:02.503840+0000","last_undegraded":"2024-09-16T14:07:02.503840+0000","last_fullsized":"2024-09-16T14:07:02.503840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:15.110271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020129200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.516454+0000","last_change":"2024-09-16T14:07:02.517047+0000","last_active":"2024-09-16T14:07:02.516454+0000","last_peered":"2024-09-16T14:07:02.516454+0000","last_clean":"2024-09-16T14:07:02.516454+0000","last_became_active":"2024-09-16T14:07:00.472006+0000","last_became_peered":"2024-09-16T14:07:00.472006+0000","last_unstale":"2024-09-16T14:07:02.516454+0000","last_undegraded":"2024-09-16T14:07:02.516454+0000","last_fullsized":"2024-09-16T14:07:02.516454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:02.291214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082399199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.502363+0000","last_change":"2024-09-16T14:07:02.502729+0000","last_active":"2024-09-16T14:07:02.502363+0000","last_peered":"2024-09-16T14:07:02.502363+0000","last_clean":"2024-09-16T14:07:02.502363+0000","last_became_active":"2024-09-16T14:07:00.476599+0000","last_became_peered":"2024-09-16T14:07:00.476599+0000","last_unstale":"2024-09-16T14:07:02.502363+0000","last_undegraded":"2024-09-16T14:07:02.502363+0000","last_fullsized":"2024-09-16T14:07:02.502363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:38.332613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056022899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.493750+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:07:02.493750+0000","last_peered":"2024-09-16T14:07:02.493750+0000","last_clean":"2024-09-16T14:07:02.493750+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:07:02.493750+0000","last_undegraded":"2024-09-16T14:07:02.493750+0000","last_fullsized":"2024-09-16T14:07:02.493750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503836+0000","last_change":"2024-09-16T14:07:02.503836+0000","last_active":"2024-09-16T14:07:02.503836+0000","last_peered":"2024-09-16T14:07:02.503836+0000","last_clean":"2024-09-16T14:07:02.503836+0000","last_became_active":"2024-09-16T14:07:00.572358+0000","last_became_peered":"2024-09-16T14:07:00.572358+0000","last_unstale":"2024-09-16T14:07:02.503836+0000","last_undegraded":"2024-09-16T14:07:02.503836+0000","last_fullsized":"2024-09-16T14:07:02.503836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:28.110370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673887,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T14:07:00.477791+0000","last_change":"2024-09-16T14:07:00.475860+0000","last_active":"2024-09-16T14:07:00.477791+0000","last_peered":"2024-09-16T14:07:00.477791+0000","last_clean":"2024-09-16T14:07:00.477791+0000","last_became_active":"2024-09-16T14:07:00.475232+0000","last_became_peered":"2024-09-16T14:07:00.475232+0000","last_unstale":"2024-09-16T14:07:00.477791+0000","last_undegraded":"2024-09-16T14:07:00.477791+0000","last_fullsized":"2024-09-16T14:07:00.477791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.457456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.502476+0000","last_change":"2024-09-16T14:07:02.502802+0000","last_active":"2024-09-16T14:07:02.502476+0000","last_peered":"2024-09-16T14:07:02.502476+0000","last_clean":"2024-09-16T14:07:02.502476+0000","last_became_active":"2024-09-16T14:07:00.480643+0000","last_became_peered":"2024-09-16T14:07:00.480643+0000","last_unstale":"2024-09-16T14:07:02.502476+0000","last_undegraded":"2024-09-16T14:07:02.502476+0000","last_fullsized":"2024-09-16T14:07:02.502476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:52.536849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:07:02.503387+0000","last_change":"2024-09-16T14:07:02.503755+0000","last_active":"2024-09-16T14:07:02.503387+0000","last_peered":"2024-09-16T14:07:02.503387+0000","last_clean":"2024-09-16T14:07:02.503387+0000","last_became_active":"2024-09-16T14:07:00.569748+0000","last_became_peered":"2024-09-16T14:07:00.569748+0000","last_unstale":"2024-09-16T14:07:02.503387+0000","last_undegraded":"2024-09-16T14:07:02.503387+0000","last_fullsized":"2024-09-16T14:07:02.503387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:11.927535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015626559999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T14:07:01.486533+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:07:01.486533+0000","last_peered":"2024-09-16T14:07:01.486533+0000","last_clean":"2024-09-16T14:07:01.486533+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:07:01.486533+0000","last_undegraded":"2024-09-16T14:07:01.486533+0000","last_fullsized":"2024-09-16T14:07:01.486533+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301788,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070052,"statfs":{"total":96636764160,"available":96327733248,"internally_reserved":0,"allocated":278528,"data_stored":664294,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":73945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297560,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074280,"statfs":{"total":96636764160,"available":96332062720,"internally_reserved":0,"allocated":208896,"data_stored":68106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4750000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":278528,"data_stored":664294,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":221184,"data_stored":73926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:07:07.544 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T14:07:07.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T14:07:07.825 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 22 2024-09-16T14:07:07.825 INFO:teuthology.orchestra.run.smithi079.stdout:fsid a22b38a6-94db-470d-bf35-45d7605f3bb0 2024-09-16T14:07:07.825 INFO:teuthology.orchestra.run.smithi079.stdout:created 2024-09-16T14:06:40.573726+0000 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:modified 2024-09-16T14:07:02.482445+0000 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:crush_version 5 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:full_ratio 0.95 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:backfillfull_ratio 0.9 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:nearfull_ratio 0.85 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:require_min_compat_client luminous 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:min_compat_client jewel 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:require_osd_release quincy 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:stretch_mode_enabled false 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-16T14:07:07.826 INFO:teuthology.orchestra.run.smithi079.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout: removed_snaps_queue [2~1] 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:max_osd 8 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/22196,v1:172.21.15.79:6817/22196] [v2:172.21.15.79:6818/22196,v1:172.21.15.79:6819/22196] exists,up a060b161-fd5a-4cfb-bac8-f2acc32d7955 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/22195,v1:172.21.15.79:6809/22195] [v2:172.21.15.79:6810/22195,v1:172.21.15.79:6811/22195] exists,up f5ba4b39-ceb8-46b3-938e-57249ed84b87 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6823/22197,v1:172.21.15.79:6825/22197] [v2:172.21.15.79:6826/22197,v1:172.21.15.79:6827/22197] exists,up 8b657f92-66c9-46a3-a31f-0bd0be414e4e 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/22193,v1:172.21.15.79:6801/22193] [v2:172.21.15.79:6802/22193,v1:172.21.15.79:6803/22193] exists,up 15937bd7-8964-4140-9f86-b3093c318d1e 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/22362,v1:172.21.15.164:6809/22362] [v2:172.21.15.164:6810/22362,v1:172.21.15.164:6811/22362] exists,up 714baf80-ade0-4e88-ad4e-5171add1fc6a 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/22363,v1:172.21.15.164:6801/22363] [v2:172.21.15.164:6802/22363,v1:172.21.15.164:6803/22363] exists,up e5a21b84-81ab-45da-a081-3b504f5c4f3d 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/22364,v1:172.21.15.164:6825/22364] [v2:172.21.15.164:6826/22364,v1:172.21.15.164:6827/22364] exists,up a009857f-f798-4154-9ff4-f524f60b6c61 2024-09-16T14:07:07.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/22365,v1:172.21.15.164:6817/22365] [v2:172.21.15.164:6818/22365,v1:172.21.15.164:6819/22365] exists,up e3506002-ba8b-4251-8ef1-136334574a6d 2024-09-16T14:07:07.839 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/22196,v1:172.21.15.79:6817/22196] [v2:172.21.15.79:6818/22196,v1:172.21.15.79:6819/22196] exists,up a060b161-fd5a-4cfb-bac8-f2acc32d7955', 'osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/22195,v1:172.21.15.79:6809/22195] [v2:172.21.15.79:6810/22195,v1:172.21.15.79:6811/22195] exists,up f5ba4b39-ceb8-46b3-938e-57249ed84b87', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6823/22197,v1:172.21.15.79:6825/22197] [v2:172.21.15.79:6826/22197,v1:172.21.15.79:6827/22197] exists,up 8b657f92-66c9-46a3-a31f-0bd0be414e4e', 'osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/22193,v1:172.21.15.79:6801/22193] [v2:172.21.15.79:6802/22193,v1:172.21.15.79:6803/22193] exists,up 15937bd7-8964-4140-9f86-b3093c318d1e', 'osd.4 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/22362,v1:172.21.15.164:6809/22362] [v2:172.21.15.164:6810/22362,v1:172.21.15.164:6811/22362] exists,up 714baf80-ade0-4e88-ad4e-5171add1fc6a', 'osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/22363,v1:172.21.15.164:6801/22363] [v2:172.21.15.164:6802/22363,v1:172.21.15.164:6803/22363] exists,up e5a21b84-81ab-45da-a081-3b504f5c4f3d', 'osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/22364,v1:172.21.15.164:6825/22364] [v2:172.21.15.164:6826/22364,v1:172.21.15.164:6827/22364] exists,up a009857f-f798-4154-9ff4-f524f60b6c61', 'osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/22365,v1:172.21.15.164:6817/22365] [v2:172.21.15.164:6818/22365,v1:172.21.15.164:6819/22365] exists,up e3506002-ba8b-4251-8ef1-136334574a6d'] 2024-09-16T14:07:07.839 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T14:07:07.936 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:07.936 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mon.a", 2024-09-16T14:07:07.936 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster": "ceph", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "admin_socket_mode": "", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_client_required": "cephx, none", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_cluster_required": "cephx", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_debug": "true", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_service_required": "cephx", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_supported": "cephx", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio": "true", 2024-09-16T14:07:07.937 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_reap_max": "16", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_async_discard": "false", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_block_size": "4096", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio": "false", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_enable_discard": "false", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_flock_retry": "3", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_inject_crash": "0", 2024-09-16T14:07:07.938 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring": "false", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring_hipri": "false", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_type": "", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_allocator": "hybrid", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_buffered_io": "true", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T14:07:07.939 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_replay_recovery": "false", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T14:07:07.940 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_sync_write": "false", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_allocator": "hybrid", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_async_db_compaction": "true", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T14:07:07.941 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_create": "true", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_create": "false", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_path": "", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_size": "0", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_path": "", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_size": "107374182400", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_create": "false", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_path": "", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs": "true", 2024-09-16T14:07:07.942 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_autotune": "true", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T14:07:07.943 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size": "0", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_type": "2q", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_clone_cow": "true", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T14:07:07.944 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_mode": "none", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_freelist": "false", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-16T14:07:07.945 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T14:07:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fail_eio": "false", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T14:07:07.947 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T14:07:07.948 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size": "0", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T14:07:07.949 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T14:07:07.950 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_mem": "512", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T14:07:07.951 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_tracing": "false", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T14:07:07.952 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T14:07:07.953 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_cluster_require_version": "2", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_require_signatures": "false", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_require_version": "2", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_service_require_signatures": "false", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_service_require_version": "2", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_sign_messages": "true", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "chdir": "", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_acl_type": "", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_asio_thread_count": "2", 2024-09-16T14:07:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_cache_mid": "0.750000", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_cache_size": "16384", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_caps_release_delay": "5", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_check_pool_perm": "true", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_force_sync_read": "false", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_getattr_caps": "false", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_inject_features": "", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T14:07:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_die_on_failed_remount": "false", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_dirsize_rbytes": "true", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_force_lazyio": "false", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_fs": "", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_inject_release_failure": "false", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_max_inline_size": "4096", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mds_namespace": "", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_metadata": "", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_gid": "-1", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_timeout": "300", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_uid": "-1", 2024-09-16T14:07:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mountpoint": "/", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_notify_timeout": "10", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc": "true", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_objects": "1000", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_size": "209715200", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_permissions": "true", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_quota": "true", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_quota_df": "true", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_max_bytes": "0", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_max_periods": "4", 2024-09-16T14:07:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_min": "131072", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_reconnect_stale": "false", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_shutdown_timeout": "30", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_snapdir": ".snap", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_tick_interval": "1", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_trace": "", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_use_faked_inos": "false", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_use_random_mds": "false", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog": "false", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog_port": "12201", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_monitors": "default=true", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog": "false", 2024-09-16T14:07:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog_level": "info", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "-", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_network": "", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_network_interface": "", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_isal": "false", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_level": "5", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zstd_level": "1", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T14:07:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location_hook": "", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location_hook_timeout": "10", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "daemonize": "false", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_alienstore": "0/5", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asok": "1/5", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asok_assert_abort": "false", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_auth": "20/20", 2024-09-16T14:07:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bdev": "1/3", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bluefs": "1/5", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bluestore": "1/5", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_buffer": "0/1", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cephsqlite": "0/5", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_civetweb": "1/10", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_client": "0/5", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_compressor": "1/5", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_context": "0/1", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crush": "1/1", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crypto": "1/5", 2024-09-16T14:07:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_dpdk": "1/5", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_eventtrace": "1/5", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_filer": "0/1", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_filestore": "1/3", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_finisher": "1/1", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_fuse": "1/5", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_javaclient": "1/5", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_journal": "1/3", 2024-09-16T14:07:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_journaler": "0/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_kstore": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_leveldb": "4/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_lockdep": "0/1", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mclock": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_balancer": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_locker": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_log": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_migrator": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_memdb": "4/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mgr": "2/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mgrc": "1/5", 2024-09-16T14:07:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mon": "20/20", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_monc": "0/10", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_ms": "1/1", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_none": "0/5", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objclass": "0/5", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objectcacher": "0/5", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objecter": "0/1", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_optracker": "0/5", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_osd": "1/5", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_paxos": "20/20", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_perfcounter": "1/5", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_prioritycache": "1/5", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rados": "0/5", 2024-09-16T14:07:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd": "0/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_replay": "0/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_refs": "0/0", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_reserver": "1/1", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw": "1/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_sync": "1/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rocksdb": "4/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore": "0/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_cache": "0/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T14:07:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_device": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_journal": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_lba": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_odata": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_omap": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_onode": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_tm": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_striper": "0/1", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_test": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_throttle": "1/1", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_timer": "0/1", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_tp": "0/5", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "device_failure_prediction_mode": "none", 2024-09-16T14:07:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_graylog": "false", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_journald": "false", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_stderr": "true", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_syslog": "false", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event_tracing": "false", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_http_port": "9926", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_prio_limit": "5", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_sort_metrics": "true", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_stats_period": "5", 2024-09-16T14:07:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "fake_statfs_for_testing": "0", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "fatal_signal_handlers": "true", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filer_max_purge_ops": "10", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filer_max_truncate_ops": "128", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_blackhole": "false", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_btrfs_snap": "true", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_caller_concurrency": "10", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T14:07:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_omap_check": "false", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_verify_split": "false", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_dump_file": "", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fadvise": "true", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fail_eio": "true", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fd_cache_size": "128", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fiemap": "false", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T14:07:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_inject_stall": "0", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_parallel": "false", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_trailing": "false", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_writeahead": "false", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_kill_at": "0", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T14:07:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_merge_threshold": "-10", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_odsync_write": "false", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_backend_path": "", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T14:07:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_threads": "2", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_punch_hole": "false", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T14:07:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_ops": "50", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_seek_data_hole": "false", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_sloppy_crc": "false", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_splice": "false", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_split_multiple": "2", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_split_rand_factor": "20", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_update_to": "1000", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T14:07:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_xfs_extsize": "false", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_zfs_snap": "false", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "fio_dir": "/tmp/fio", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "a22b38a6-94db-470d-bf35-45d7605f3bb0", 2024-09-16T14:07:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_allow_other": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_big_writes": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_debug": "false", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_default_permissions": "false", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_disable_pagecache": "false", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_max_write": "0", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_multithreaded": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_require_active_mds": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_set_user_groups": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_move": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_read": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_write": "true", 2024-09-16T14:07:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "gss_target_name": "ceph", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_file": "", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_inject_failure": "0", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_interval": "5", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "host": "smithi079", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T14:07:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "inject_early_sigterm": "false", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "jaeger_tracing_enable": "false", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_aio": "true", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_align_min_size": "65536", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_block_align": "true", 2024-09-16T14:07:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_block_size": "4096", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_dio": "true", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_discard": "false", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_force_aio": "false", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_ignore_corruption": "false", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_max_write_entries": "100", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_replay_from": "0", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_write_header_frequency": "0", 2024-09-16T14:07:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_zero_on_create": "false", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_prefetch_periods": "10", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_prezero_periods": "5", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_write_head_interval": "15", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "keyfile": "", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_backend": "rocksdb", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_max_bytes": "67108864", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_max_ops": "512", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T14:07:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_onode_map_size": "1024", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_sync_transaction": "false", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_block_size": "65536", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_bloom_size": "0", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_cache_size": "536870912", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_compact_on_mount": "false", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_compression": "false", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_log": "", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_max_open_files": "0", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_paranoid": "false", 2024-09-16T14:07:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "librados_thread_count": "2", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "lockdep": "false", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "lockdep_force_backtrace": "false", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_coarse_timestamps": "true", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_flush_on_exit": "false", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_graylog_port": "12201", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_max_new": "1000", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_max_recent": "10000", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_stderr_prefix": "", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T14:07:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": "true", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_graylog": "false", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_journald": "false", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_stderr": "false", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_syslog": "false", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_action_on_write_error": "1", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_alternate_name_max": "8192", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_asio_thread_count": "2", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_export_pin": "true", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T14:07:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_interval": "10", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_max": "-1", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_max_until": "-1", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_merge_size": "50", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_mode": "0", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T14:07:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_bits": "3", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_size": "10000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T14:07:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_mid": "0.700000", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_interval": "1", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T14:07:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_auth_pins": "false", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_frag": "false", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_scatterstat": "false", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_subtrees": "false", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_default_dir_hash": "2", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_defer_session_stale": "true", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T14:07:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_max_entries": "0", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_early_reply": "true", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_enable_op_tracker": "true", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_enforce_unique_name": "true", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T14:07:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_health_dummy": "false", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T14:07:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_join_fs": "", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_journal_format": "1", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_create_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_export_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_import_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_link_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_openc_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_rename_at": "0", 2024-09-16T14:07:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_max_events": "-1", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_max_segments": "128", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_pause": "false", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_segment_size": "0", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_completed_requests": "100000", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_export_size": "20971520", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_file_recover": "32", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_files": "64", 2024-09-16T14:07:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_ops": "8192", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_metrics_update_interval": "2", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_min_caps_per_client": "100", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_numa_node": "-1", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_duration": "600", 2024-09-16T14:07:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_size": "20", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_log_threshold": "5", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_ping_grace": "15", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_ping_interval": "5", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_caps": "30000", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T14:07:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_replay_interval": "1.000000", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_root_ino_gid": "0", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_root_ino_uid": "0", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T14:07:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_shutdown_check": "0", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_skip_ino": "0", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_min_uid": "0", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_rstat": "false", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_symlink_recovery": "true", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_thrash_exports": "0", 2024-09-16T14:07:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_thrash_fragments": "0", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_tick_interval": "5.000000", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_valgrind_exit": "false", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_verify_backtrace": "1", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_verify_scatter": "false", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_wipe_sessions": "false", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool_debug": "false", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_page_set": "false", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_page_size": "65536", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_bytes": "134217728", 2024-09-16T14:07:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_messages": "512", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mds_messages": "128", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T14:07:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mon_messages": "128", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_osd_messages": "8192", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_pool": "true", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_standby_modules": "true", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_stats_period": "5", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_stats_threshold": "5", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_tick_period": "2", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_allow_pool_delete": "true", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T14:07:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_bytes": "104857600", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_directed_command_retry": "5", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T14:07:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_level": "debug", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T14:07:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_start": "false", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_trim": "true", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cpu_threads": "4", 2024-09-16T14:07:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_avail_crit": "5", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_avail_warn": "5", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_json": "false", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_extra_checks": "false", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T14:07:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_pacific": "false", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_quincy": "false", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_down_mkfs_grace": "120", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_election_default_strategy": "1", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_election_timeout": "5.000000", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_enable_op_tracker": "true", 2024-09-16T14:07:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_fake_pool_delete": "false", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_force_quorum_join": "false", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_log_update_period": "5", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_max_detail": "50", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog": "true", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_host": "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.164", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_host_override": "", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_initial_members": "", 2024-09-16T14:07:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease": "5.000000", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_full_interval": "50", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_max": "10000", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_max_summary": "50", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_log_epochs": "500", 2024-09-16T14:07:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_osd": "10000", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_memory_autotune": "true", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_memory_target": "2147483648", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T14:07:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_digest_period": "5", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_complaint_time": "30", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_duration": "600", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_size": "20", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T14:07:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_log_threshold": "5", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_cache_size": "500", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T14:07:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T14:07:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_report_timeout": "900", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T14:07:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_max_osds": "4", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T14:07:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_interval": "86400", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_max_keys": "100", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_timeout": "300", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_session_timeout": "300", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_smart_report_timeout": "5", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_pool_size": "4", 2024-09-16T14:07:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_debug": "false", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_tick_interval": "5", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T14:07:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_older_version": "true", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T14:07:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "monmap": "", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_op_threads": "3", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_cm": "false", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_device_name": "", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T14:07:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_sl": "3", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_type": "ib", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_reap_threshold": "5", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_before_connect": "false", 2024-09-16T14:07:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_ipv4": "true", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_ipv6": "false", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_msgr1": "true", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_msgr2": "true", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_port_max": "7568", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_port_min": "6800", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_retry_count": "3", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_retry_delay": "5", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_client": "false", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mds": "false", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mgr": "false", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mon": "false", 2024-09-16T14:07:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_osd": "false", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_client_mode": "crc secure", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_cluster_type": "", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_compress_secure": "false", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_crc_data": "true", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_crc_header": "true", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_bug": "true", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_old_message": "true", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T14:07:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hugepages": "", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_lro": "true", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T14:07:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_pmd": "", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_port_id": "0", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dump_on_send": "false", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_type": "", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_socket_failures": "5000", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T14:07:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_max_accept_failures": "4", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_max_backoff": "15.000000", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compress_mode": "none", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_pq_min_cost": "65536", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_public_type": "", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_service_mode": "crc secure", 2024-09-16T14:07:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_nodelay": "true", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_type": "async+posix", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "no_config_file": "false", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inflight_ops": "1024", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T14:07:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_timeout": "10.000000", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "objectstore_blackhole": "false", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "openssl_engine_opts": "", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_max_ops": "4", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_slop": "0.020000", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_asio_thread_count": "2", 2024-09-16T14:07:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_scan_max": "512", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_scan_min": "64", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_peering": "false", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_beacon_report_interval": "300", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_duration": "30", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T14:07:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_blkin_trace_all": "false", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_update_on_start": "true", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_message_cap": "256", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_op_priority": "63", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_watch_timeout": "30", 2024-09-16T14:07:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_command_max_records": "256", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_compact_on_start": "false", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_update_on_start": "true", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T14:07:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_op_order": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_shutdown": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T14:07:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_default_notify_timeout": "30", 2024-09-16T14:07:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_enable_op_tracker": "true", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_info": "true", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown": "true", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T14:07:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_function_tracing": "false", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_grace": "20", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_interval": "6", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_stale": "600", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T14:07:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal_size": "5120", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_kill_backfill_at": "0", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_cache_size": "50", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_dedup": "true", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_message_max": "40", 2024-09-16T14:07:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_attr_name_len": "100", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_attr_size": "0", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_backfills": "1", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_markdown_count": "5", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_markdown_period": "600", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_name_len": "2048", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_size": "134217728", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T14:07:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pgls": "1024", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_push_cost": "8388608", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_push_objects": "10", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_scrubs": "3", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_write_size": "90", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T14:07:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_profile": "balanced", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T14:07:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_base": "805306368", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target": "4294967296", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target_autotune": "false", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_min_recovery_priority": "0", 2024-09-16T14:07:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_report_interval": "5", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_num_cache_shards": "32", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_node": "-1", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objecter_finishers": "1", 2024-09-16T14:07:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore": "bluestore", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore_fuse": "false", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore_tracing": "false", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_duration": "600", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_size": "20", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_log_threshold": "5", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards": "0", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T14:07:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_thread_timeout": "15", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_open_classes_on_start": "true", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_os_flags": "0", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_peering_op_priority": "255", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_delete_priority": "5", 2024-09-16T14:07:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T14:07:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flags": "0", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_min_size": "0", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T14:07:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_size": "2", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_type": "replicated", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_cost": "20971520", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T14:07:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active": "0", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_op_priority": "3", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_priority": "5", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T14:07:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_repair_during_recovery": "false", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_cost": "52428800", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T14:07:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_end_hour": "0", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_priority": "5", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_skip_data_digest": "false", 2024-09-16T14:07:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_smart_report_timeout": "5", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_priority": "5", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_target_transaction_size": "30", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T14:07:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tracing": "false", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_use_stale_snap": "false", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_kill_at": "0", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_max_join_drift": "10", 2024-09-16T14:07:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_min": "500", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_min_wait": "0.050000", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_max": "500", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_min": "250", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_stash_full_interval": "25", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_trim_max": "500", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_trim_min": "250", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "perf": "true", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-16T14:07:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "-", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrv": "", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "public_bind_addr": "-", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "public_network": "", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "public_network_interface": "", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "qat_compressor_enabled": "false", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_mon_op_timeout": "0", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_osd_op_timeout": "0", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_tracing": "false", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_atime_update_interval": "60", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T14:07:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache": "true", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_policy": "writearound", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_size": "33554432", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T14:07:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_compression_hint": "none", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_clone_format": "auto", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_data_pool": "", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_features": "61", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_format": "2", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_map_options": "", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_order": "22", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_pool": "rbd", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_stripe_count": "0", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T14:07:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_io_scheduler": "simple", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T14:07:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_order": "24", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_pool": "", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_splay_width": "4", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T14:07:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T14:07:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T14:07:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_op_threads": "1", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_plugins": "", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T14:07:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T14:07:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_tracing": "false", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_validate_names": "true", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_validate_pool": "true", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "restapi_base_url": "", 2024-09-16T14:07:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "restapi_log_level": "", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_admin_entry": "admin", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_backend_store": "rados", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_barbican_url": "", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_beast_enable_async": "true", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_enabled": "true", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_lru_size": "10000", 2024-09-16T14:07:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_content_length_compat": "false", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cross_domain_policy": "", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T14:07:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T14:07:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T14:07:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T14:07:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_window": "30", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T14:07:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T14:07:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dns_name": "", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dns_s3website_name": "", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_ops_log": "false", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_static_website": "false", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_usage_log": "false", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T14:07:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_expose_bucket": "false", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_extended_http_attrs": "", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_objs": "32", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T14:07:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_init_timeout": "300", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_kafka_connection_idle": "300", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T14:07:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_password": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_project": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_token": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_user": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_api_version": "2", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T14:07:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_url": "", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_objs": "32", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T14:07:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_worker": "3", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_uri": "", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_http_headers": "", 2024-09-16T14:07:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attr_size": "0", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_listing_results": "1000", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_notify_retries": "10", 2024-09-16T14:07:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T14:07:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_num_control_oids": "8", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_numa_node": "-1", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T14:07:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_tracing": "false", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_token": "", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_url": "", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_file_path": "", 2024-09-16T14:07:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_rados": "true", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_print_continue": "true", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T14:07:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_tracing": "false", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm": "", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm_id": "", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_region": "", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_request_uri": "", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T14:07:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_resolve_cname": "false", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_run_sync_thread": "true", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T14:07:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_script_uri": "", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_service_provider_name": "", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_client_id": "", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_client_secret": "", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_entry": "sts", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_key": "sts", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T14:07:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_auth_url": "", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_custom_header": "", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_need_stats": "true", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_tenant_name": "", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_url": "", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_lease_period": "120", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T14:07:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_thread_pool_size": "512", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_comment": "", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_createby": "", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_encoding": "", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_flag": "false", 2024-09-16T14:07:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_origin": "", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_tracker": "", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_max_shards": "32", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_use_opa_authz": "false", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T14:07:08.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_unique_email": "true", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_verify_ssl": "true", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone": "", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone_id": "", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup": "", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup_id": "", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T14:07:08.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_block_size": "4096", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T14:07:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_partition_filters": "false", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_perf": "false", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "run_dir": "/var/run/ceph", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_block_create": "true", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T14:07:08.083 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_device_size": "107374182400", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_segment_size": "67108864", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "setgroup": "", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "setuser": "", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "setuser_match_path": "", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "thp": "false", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "threadpool_default_timeout": "60", 2024-09-16T14:07:08.084 INFO:teuthology.orchestra.run.smithi079.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T14:07:08.085 INFO:teuthology.orchestra.run.smithi079.stdout: "throttler_perf_counter": "true" 2024-09-16T14:07:08.085 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:08.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T14:07:08.226 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a: {} 2024-09-16T14:07:08.239 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b: {} 2024-09-16T14:07:08.253 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c: {} 2024-09-16T14:07:08.255 INFO:teuthology.orchestra.run.smithi079.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:07:08.256 INFO:teuthology.orchestra.run.smithi079.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:07:08.256 INFO:teuthology.orchestra.run.smithi079.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:07:08.270 DEBUG:teuthology.orchestra.run.smithi164:> type ceph-objectstore-tool 2024-09-16T14:07:08.276 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T14:07:08.276 DEBUG:teuthology.orchestra.run.smithi079:> type ceph-objectstore-tool 2024-09-16T14:07:08.282 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T14:07:08.282 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-16T14:07:08.291 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-16T14:07:08.292 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T14:07:08.292 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:07:08.292 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:07:08.292 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T14:07:08.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-16T14:07:08.293 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T14:07:08.293 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T14:07:08.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:08.294 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T14:07:08.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:08.294 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T14:07:08.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:07:08.295 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-16T14:07:08.296 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:08.290+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:08.320 INFO:teuthology.orchestra.run.smithi079.stdout:mapped writes DISABLED 2024-09-16T14:07:08.320 INFO:teuthology.orchestra.run.smithi079.stdout:mapped reads DISABLED 2024-09-16T14:07:08.320 INFO:teuthology.orchestra.run.smithi079.stdout:Seed set to 2840 2024-09-16T14:07:08.395 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:08.391+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:08.426 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:08.427 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:08.427 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:08.427 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:08.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:08.462 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:08.485 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:08.495 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:08.490+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:08.503 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:08.522 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:08.545 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:08.546 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:08.552 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:08.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:08.572 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:08.593 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:08.596 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:08.590+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:08.626 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:08.628 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:08.628 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:08.629 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:08.636 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 82 (0%) 2024-09-16T14:07:08.636 INFO:teuthology.orchestra.run.smithi079.stdout:avg 10.25 2024-09-16T14:07:08.636 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-16T14:07:08.636 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-16T14:07:08.636 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-16T14:07:08.636 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:08.636 INFO:teuthology.orchestra.run.smithi079.stdout:oload 110 2024-09-16T14:07:08.636 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 1 2024-09-16T14:07:08.637 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-16T14:07:08.637 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0032 2024-09-16T14:07:08.637 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0035 2024-09-16T14:07:08.639 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-16T14:07:08.649 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T14:07:08.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-16T14:07:08.652 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:08.652 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:08.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:08.697 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:08.695+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:08.797 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:08.797 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:08.798 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:08.794+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:08.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:08.898 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:08.895+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:08.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 82 (0%) 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:avg 10.25 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:oload 110 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 0.05 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0032 2024-09-16T14:07:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0035 2024-09-16T14:07:08.924 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-16T14:07:08.934 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T14:07:08.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-16T14:07:08.999 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:08.994+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:09.004 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.004 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:09.100 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:09.094+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:09.163 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.163 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:09.201 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:09.195+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 82 (0%) 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:avg 10.25 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:oload 110 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 0.05 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0032 2024-09-16T14:07:09.237 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0035 2024-09-16T14:07:09.241 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-16T14:07:09.250 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T14:07:09.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-16T14:07:09.271 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.271 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:09.302 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:09.299+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:09.374 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.375 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:09.403 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:09.398+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:09.480 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.480 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:09.504 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:09.499+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:09.552 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:07:09.582 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 82 (0%) 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:avg 10.25 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:oload 110 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 1 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0032 2024-09-16T14:07:09.583 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0035 2024-09-16T14:07:09.587 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-16T14:07:09.600 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T14:07:09.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-16T14:07:09.604 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:09.599+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:09.649 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.649 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:09.705 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:09.698+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:09.756 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.756 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:09.812 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:09.807+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:09.874 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.874 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.874 INFO:teuthology.orchestra.run.smithi079.stdout:moved 0 / 82 (0%) 2024-09-16T14:07:09.874 INFO:teuthology.orchestra.run.smithi079.stdout:avg 10.25 2024-09-16T14:07:09.874 INFO:teuthology.orchestra.run.smithi079.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-16T14:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-16T14:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-16T14:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout:oload 110 2024-09-16T14:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout:max_change 0.05 2024-09-16T14:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout:max_change_osds 4 2024-09-16T14:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout:average_utilization 0.0032 2024-09-16T14:07:09.875 INFO:teuthology.orchestra.run.smithi079.stdout:overload_utilization 0.0035 2024-09-16T14:07:09.877 INFO:teuthology.orchestra.run.smithi079.stderr:no change 2024-09-16T14:07:09.883 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:09.907 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:09.903+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:09.985 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:09.985 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:09.994 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:10.008 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:10.002+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:10.109 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:10.103+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:10.139 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:10.140 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:10.148 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:10.211 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:10.203+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:10.249 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:10.249 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:10.258 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:10.311 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:10.306+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:10.402 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:10.402 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:10.411 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:10.412 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:10.407+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:10.508 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:10.508 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:10.513 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:10.507+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:10.517 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:10.615 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:10.607+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:10.642 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:10.667 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:10.667 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:10.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:10.715 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:10.711+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:10.777 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:10.778 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:10.788 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:10.790 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:10.808 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:10.831 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:10.827+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:10.917 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:10.914+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:10.936 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:10.936 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:10.945 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:11.018 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:11.011+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:11.036 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:11.053 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:11.053 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:11.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:11.063 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:11.081 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:11.102 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:11.119 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:11.115+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:11.123 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:11.144 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:11.147 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:11.147 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:11.147 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:11.147 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:11.147 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:11.147 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:11.147 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:11.148 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:11.162 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:11.162 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:11.163 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:11.163 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:11.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:11.221 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:11.215+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:11.269 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:11.269 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:11.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:11.269 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [] 2024-09-16T14:07:11.269 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:11.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:11.321 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:11.314+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:11.396 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:11.396 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:11.396 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:11.396 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:11.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:11.450 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:11.446+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:11.523 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:11.519+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:11.553 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:11.553 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:11.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:11.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:07:11.624 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:11.618+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.348446+0000", 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.30755671099999998, 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045895969999999999, 2024-09-16T14:07:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348446+0000", 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348446+0000", 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348446+0000", 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.77e-07 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348452+0000", 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9440000000000002e-06 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348477+0000", 2024-09-16T14:07:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4967e-05 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348486+0000", 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.799e-06 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348561+0000", 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5389999999999995e-05 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348595+0000", 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4235e-05 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.352954+0000", 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043584519999999996 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.353035+0000", 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1433000000000005e-05 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:12 3.e e26/23 3:7276ff7f:::rbd_header.113631a28a63:head v 26'1, mlcod=0'0)", 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.361703+0000", 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.29429941500000001, 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001438078, 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361703+0000", 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361703+0000", 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361703+0000", 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2e-07 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361711+0000", 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8520000000000004e-06 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361748+0000", 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6844000000000002e-05 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361763+0000", 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5017999999999999e-05 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361914+0000", 2024-09-16T14:07:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000151199 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361963+0000", 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8401e-05 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363058+0000", 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001095387 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363141+0000", 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2856999999999996e-05 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:25 3.e e26/23 3:7276ff7f:::rbd_header.113631a28a63:head v 26'2, mlcod=0'0)", 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.388822+0000", 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.26718040399999998, 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001015501, 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388822+0000", 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388822+0000", 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388824+0000", 2024-09-16T14:07:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.069e-06 2024-09-16T14:07:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388843+0000", 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8779000000000001e-05 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388886+0000", 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3605e-05 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388903+0000", 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6356999999999999e-05 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388989+0000", 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6154000000000002e-05 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389017+0000", 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8212999999999999e-05 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389721+0000", 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000704476 2024-09-16T14:07:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389837+0000", 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011584800000000001 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:11.672 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:11.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:11.724 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:11.719+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:11.774 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:11.774 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:11.774 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:11.782 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:11.825 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:11.819+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:11.880 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:11.880 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:11.880 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:11.880 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:11.880 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:11.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:11.926 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:11.922+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:11.998 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:11.998 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:11.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:11.998 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:11.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.145127459000001, 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15070547300000001, 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:11.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847226+0000", 2024-09-16T14:07:12.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.204e-06 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847235+0000", 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847250+0000", 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5662e-05 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847261+0000", 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0236e-05 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847327+0000", 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.622e-05 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:12.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847345+0000", 2024-09-16T14:07:12.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8434e-05 2024-09-16T14:07:12.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.993706+0000", 2024-09-16T14:07:12.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14636041899999999 2024-09-16T14:07:12.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994117+0000", 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041132499999999998 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994428+0000", 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031060899999999999 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.995845+0000", 2024-09-16T14:07:12.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014173359999999999 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997768+0000", 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001922991 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997867+0000", 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9057999999999999e-05 2024-09-16T14:07:12.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997929+0000", 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2433000000000003e-05 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.225136+0000", 2024-09-16T14:07:12.007 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.767215330999999, 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010268787, 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 42 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225136+0000", 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225136+0000", 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225137+0000", 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4499999999999996e-07 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225141+0000", 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5600000000000004e-06 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225150+0000", 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2889999999999998e-06 2024-09-16T14:07:12.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225155+0000", 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9110000000000001e-06 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.228294+0000", 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003139102 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.228355+0000", 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0863999999999999e-05 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.228460+0000", 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105392 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.232739+0000", 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042787149999999998 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235331+0000", 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002592759 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235373+0000", 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1325999999999997e-05 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235405+0000", 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2023999999999997e-05 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.014 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226025+0000", 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.766326118999999, 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010082832999999999, 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45 2024-09-16T14:07:12.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226025+0000", 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226025+0000", 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226027+0000", 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4309999999999999e-06 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226035+0000", 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2230000000000007e-06 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226058+0000", 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2878999999999999e-05 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226069+0000", 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1073999999999999e-05 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.231642+0000", 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055735689999999996 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.231751+0000", 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108142 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.231902+0000", 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015110300000000001 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235420+0000", 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035178520000000001 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235943+0000", 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052378799999999997 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236043+0000", 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9788000000000001e-05 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236108+0000", 2024-09-16T14:07:12.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4984e-05 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226091+0000", 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.766260481, 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011816414000000001, 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 46 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226091+0000", 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226091+0000", 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226092+0000", 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.08e-06 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226102+0000", 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226124+0000", 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2282000000000001e-05 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226146+0000", 2024-09-16T14:07:12.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2087999999999999e-05 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.233144+0000", 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069974499999999997 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.233335+0000", 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019087600000000001 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.233599+0000", 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026448699999999999 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235852+0000", 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022531270000000002 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237788+0000", 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00193542 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237847+0000", 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8962999999999998e-05 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237907+0000", 2024-09-16T14:07:12.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0751999999999997e-05 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226215+0000", 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.766136318000001, 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011817500999999999, 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 47 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226215+0000", 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226215+0000", 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226217+0000", 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5149999999999999e-06 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226403+0000", 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018652100000000001 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226429+0000", 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6363000000000002e-05 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226447+0000", 2024-09-16T14:07:12.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7342000000000001e-05 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.234097+0000", 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007650034 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.234228+0000", 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000131576 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.234432+0000", 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020386800000000001 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236598+0000", 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021659919999999998 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237965+0000", 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013668770000000001 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238005+0000", 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0108999999999998e-05 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238033+0000", 2024-09-16T14:07:12.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7304000000000002e-05 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.765875825999998, 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013812964, 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.036 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 48 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226477+0000", 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.879999999999999e-07 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226482+0000", 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0919999999999997e-06 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226537+0000", 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4916000000000001e-05 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226546+0000", 2024-09-16T14:07:12.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9709999999999999e-06 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236148+0000", 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0096011849999999999 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236208+0000", 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9922e-05 2024-09-16T14:07:12.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236308+0000", 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010068000000000001 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239183+0000", 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002874779 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240208+0000", 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001025312 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240255+0000", 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6363999999999999e-05 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240288+0000", 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3754999999999998e-05 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.765796725000001, 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013937603, 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.043 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 49 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0299999999999998e-07 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226560+0000", 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5879999999999999e-06 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226568+0000", 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3799999999999994e-06 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226575+0000", 2024-09-16T14:07:12.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236676+0000", 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010100448999999999 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236851+0000", 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017497899999999999 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237123+0000", 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027242299999999999 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240139+0000", 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030155109999999998 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240368+0000", 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000229461 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240417+0000", 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9045999999999998e-05 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240492+0000", 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5177000000000002e-05 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.765769837000001, 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015154025999999999, 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 50 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226710+0000", 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012769999999999999 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226721+0000", 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226726+0000", 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6459999999999998e-06 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238128+0000", 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011401390000000001 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238249+0000", 2024-09-16T14:07:12.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121284 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238407+0000", 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015738899999999999 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240509+0000", 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002102923 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241459+0000", 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094996999999999996 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241621+0000", 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000161502 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241736+0000", 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011458799999999999 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.765616911999999, 2024-09-16T14:07:12.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015305664, 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 51 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:12.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226735+0000", 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5700000000000001e-07 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226740+0000", 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6460000000000001e-06 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226748+0000", 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226753+0000", 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6229999999999996e-06 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239228+0000", 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012475119999999999 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239405+0000", 2024-09-16T14:07:12.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177641 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239559+0000", 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015368599999999999 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241393+0000", 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018340260000000001 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241863+0000", 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047025300000000001 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241934+0000", 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0229999999999997e-05 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242040+0000", 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106568 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.765590518, 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015737872, 2024-09-16T14:07:12.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 52 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:12.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226762+0000", 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.23e-07 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226766+0000", 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.668e-06 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226774+0000", 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8939999999999993e-06 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226779+0000", 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1170000000000003e-06 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240584+0000", 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013804344 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240709+0000", 2024-09-16T14:07:12.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012524300000000001 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241014+0000", 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030534399999999998 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242062+0000", 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010479899999999999 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242369+0000", 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030710399999999999 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242442+0000", 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2383000000000001e-05 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242499+0000", 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.770661+0000", 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.221690554999999, 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011089416, 2024-09-16T14:07:12.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 68 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770661+0000", 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770661+0000", 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770663+0000", 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3199999999999998e-06 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770673+0000", 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0216e-05 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770694+0000", 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0942999999999999e-05 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770707+0000", 2024-09-16T14:07:12.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2343e-05 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.776828+0000", 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061213450000000003 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.776928+0000", 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100012 2024-09-16T14:07:12.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.777100+0000", 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017197600000000001 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781463+0000", 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043629669999999997 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781606+0000", 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014329100000000001 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781705+0000", 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9123999999999995e-05 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781750+0000", 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4879000000000002e-05 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.770777+0000", 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.221574345000001, 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011286358999999999, 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.078 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 69 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770777+0000", 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770777+0000", 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770779+0000", 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.95e-06 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770915+0000", 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013586699999999999 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770927+0000", 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2670000000000001e-05 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770939+0000", 2024-09-16T14:07:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1487e-05 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.777750+0000", 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068105800000000001 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.777871+0000", 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012134900000000001 2024-09-16T14:07:12.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.778043+0000", 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000171953 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781843+0000", 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038004369999999998 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781864+0000", 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0831000000000001e-05 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781963+0000", 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9173999999999996e-05 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782063+0000", 2024-09-16T14:07:12.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100061 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.770966+0000", 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.221385775000002, 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012891682999999999, 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.085 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 70 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770966+0000", 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770966+0000", 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770967+0000", 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3060000000000001e-06 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770976+0000", 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3500000000000003e-06 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770995+0000", 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8756000000000001e-05 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771007+0000", 2024-09-16T14:07:12.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.187e-05 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.779351+0000", 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083437349999999997 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.779443+0000", 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2261000000000003e-05 2024-09-16T14:07:12.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.779629+0000", 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018625 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782202+0000", 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025728439999999999 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783776+0000", 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015739700000000001 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783826+0000", 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9602999999999999e-05 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783857+0000", 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1738e-05 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.22132998, 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013869546999999999, 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.092 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 71 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771022+0000", 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0699999999999999e-06 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771031+0000", 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3899999999999993e-06 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771048+0000", 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6949e-05 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771062+0000", 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4647000000000001e-05 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780217+0000", 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091543300000000005 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780323+0000", 2024-09-16T14:07:12.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106223 2024-09-16T14:07:12.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780485+0000", 2024-09-16T14:07:12.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016238299999999999 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784807+0000", 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043215500000000004 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784821+0000", 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4268e-05 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784861+0000", 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9462000000000003e-05 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784891+0000", 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0275000000000001e-05 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.221275534, 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015694993000000001, 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 72 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771077+0000", 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0299999999999997e-07 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771209+0000", 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132347 2024-09-16T14:07:12.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771224+0000", 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.454e-05 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771230+0000", 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:07:12.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782272+0000", 2024-09-16T14:07:12.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011041455 2024-09-16T14:07:12.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782402+0000", 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013039099999999999 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782645+0000", 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024303500000000001 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786509+0000", 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003863698 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786628+0000", 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011894499999999999 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786711+0000", 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2961999999999996e-05 2024-09-16T14:07:12.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786771+0000", 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0098e-05 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.107 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.221112805000001, 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016692631999999999, 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 73 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771240+0000", 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.569999999999999e-07 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771244+0000", 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771253+0000", 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8629999999999997e-06 2024-09-16T14:07:12.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771262+0000", 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783924+0000", 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012662479000000001 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784015+0000", 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0624999999999994e-05 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784129+0000", 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113563 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787598+0000", 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034694890000000001 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787707+0000", 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109341 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787840+0000", 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132421 2024-09-16T14:07:12.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787931+0000", 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1291000000000004e-05 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:12.114 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.221082709000001, 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017320314999999999, 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:12.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771278+0000", 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3790000000000006e-06 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771286+0000", 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5199999999999997e-06 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771292+0000", 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2839999999999997e-06 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784933+0000", 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013641304 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785019+0000", 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6201999999999999e-05 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785164+0000", 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014457599999999999 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787609+0000", 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024456769999999998 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788403+0000", 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079383000000000003 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788490+0000", 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7231000000000006e-05 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788589+0000", 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8608000000000002e-05 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.121 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.221052999000001, 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017489688, 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.122 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771299+0000", 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3e-07 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771421+0000", 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121544 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771433+0000", 2024-09-16T14:07:12.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853e-05 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771443+0000", 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0229999999999998e-06 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785601+0000", 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014158145 2024-09-16T14:07:12.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785677+0000", 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6711999999999993e-05 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785774+0000", 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6583999999999995e-05 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787633+0000", 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001859461 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788688+0000", 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010546349999999999 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788747+0000", 2024-09-16T14:07:12.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.923e-05 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788788+0000", 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0771e-05 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.220896418999999, 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017898397999999999, 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 76 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771456+0000", 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7900000000000007e-07 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771466+0000", 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8800000000000003e-06 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771475+0000", 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1670000000000005e-06 2024-09-16T14:07:12.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771480+0000", 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.49e-06 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786862+0000", 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015381807000000001 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786985+0000", 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012302099999999999 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787142+0000", 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015696500000000001 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788157+0000", 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001014923 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789175+0000", 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010175200000000001 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789259+0000", 2024-09-16T14:07:12.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4578000000000002e-05 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789353+0000", 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4067999999999995e-05 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T14:07:12.135 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:55.579053+0000", 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.413298206, 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010202396000000001, 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 94 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579053+0000", 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579053+0000", 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579054+0000", 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8299999999999995e-07 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579059+0000", 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0429999999999998e-06 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579080+0000", 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0740000000000001e-05 2024-09-16T14:07:12.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579086+0000", 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9630000000000003e-06 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.585180+0000", 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060944880000000003 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.585248+0000", 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7447999999999995e-05 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.585377+0000", 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012897599999999999 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.586525+0000", 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011482420000000001 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.589165+0000", 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026399010000000001 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.589202+0000", 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6711999999999997e-05 2024-09-16T14:07:12.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.589256+0000", 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3999999999999998e-05 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.142 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:12.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:12.144 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:12.138+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:12.245 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:12.238+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:12.285 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:12.285 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:12.285 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:12.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:12.345 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:12.338+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:12.388 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:12.388 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:12.388 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:12.388 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:12.389 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:12.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:12.446 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:12.442+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:12.494 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:12.494 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:12.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:12.494 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:12.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.494 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4241.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.571388+0000", 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.920222301000001, 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002857563, 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.571388+0000", 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.571388+0000", 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.571388+0000", 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0299999999999998e-07 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.571401+0000", 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2566e-05 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.571464+0000", 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3221999999999999e-05 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.571488+0000", 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999999e-05 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.571572+0000", 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3757999999999993e-05 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.571602+0000", 2024-09-16T14:07:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0969000000000001e-05 2024-09-16T14:07:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574183+0000", 2024-09-16T14:07:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00258099 2024-09-16T14:07:12.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574245+0000", 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1774999999999997e-05 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1338266379999999, 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056564060000000001, 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357785+0000", 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0779999999999998e-06 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357794+0000", 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357811+0000", 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7300999999999999e-05 2024-09-16T14:07:12.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357825+0000", 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3959e-05 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357934+0000", 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010899 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357942+0000", 2024-09-16T14:07:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.182e-06 2024-09-16T14:07:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.360955+0000", 2024-09-16T14:07:12.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003013258 2024-09-16T14:07:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361149+0000", 2024-09-16T14:07:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019401600000000001 2024-09-16T14:07:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361448+0000", 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029863300000000003 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.362788+0000", 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013406480000000001 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363338+0000", 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054941500000000004 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363400+0000", 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2440999999999994e-05 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363440+0000", 2024-09-16T14:07:12.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9734999999999998e-05 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4406.0:12 3.e e26/23 ondisk, result = 0)", 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.363197+0000", 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1284127269999999, 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025178499999999998, 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.509 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363197+0000", 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363197+0000", 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363199+0000", 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.488e-06 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363208+0000", 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2720000000000003e-06 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363236+0000", 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7911000000000001e-05 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363253+0000", 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6648000000000001e-05 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363321+0000", 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8553000000000006e-05 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363330+0000", 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5760000000000006e-06 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363449+0000", 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119337 2024-09-16T14:07:12.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:16 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.380261+0000", 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.111349374, 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00076907899999999999, 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 16 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.380261+0000", 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.380261+0000", 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.380263+0000", 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.712e-06 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.380289+0000", 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5630999999999999e-05 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.380320+0000", 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1333e-05 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.380332+0000", 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2149e-05 2024-09-16T14:07:12.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.380448+0000", 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115154 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.380598+0000", 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015004499999999999 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381030+0000", 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00043205499999999998 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:17 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.381277+0000", 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.11033253, 2024-09-16T14:07:12.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066424799999999999, 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 17 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381277+0000", 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381277+0000", 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381279+0000", 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.906e-06 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381289+0000", 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5310000000000007e-06 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381318+0000", 2024-09-16T14:07:12.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8699000000000001e-05 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381329+0000", 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.146e-05 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381433+0000", 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104178 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381643+0000", 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000209996 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.381942+0000", 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029847800000000002 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:18 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.382129+0000", 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.109480553, 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00049080399999999996, 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.525 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 18 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382129+0000", 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382129+0000", 2024-09-16T14:07:12.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382131+0000", 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.497e-06 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382138+0000", 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382157+0000", 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8960000000000001e-05 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382166+0000", 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3250000000000008e-06 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382239+0000", 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3289999999999998e-05 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382388+0000", 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014945400000000001 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382620+0000", 2024-09-16T14:07:12.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000231821 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:19 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.382799+0000", 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.108811065, 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000373178, 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 19 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382799+0000", 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382799+0000", 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382800+0000", 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.426e-06 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382807+0000", 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8079999999999999e-06 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382825+0000", 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8223000000000002e-05 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382834+0000", 2024-09-16T14:07:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2279999999999998e-06 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.382907+0000", 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3808999999999998e-05 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.383013+0000", 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010580499999999999 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.383172+0000", 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015887900000000001 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:20 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.383898+0000", 2024-09-16T14:07:12.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1077121000000001, 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075906300000000003, 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.383898+0000", 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.383898+0000", 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.383900+0000", 2024-09-16T14:07:12.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.705e-06 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.383909+0000", 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0089999999999996e-06 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.383932+0000", 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3756000000000001e-05 2024-09-16T14:07:12.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.383942+0000", 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8570000000000007e-06 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384016+0000", 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3758999999999996e-05 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384141+0000", 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012528300000000001 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384657+0000", 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051569400000000003 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:21 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.384874+0000", 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1067357929999999, 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000572742, 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2024-09-16T14:07:12.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384874+0000", 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384874+0000", 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384876+0000", 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5939999999999999e-06 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384884+0000", 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4999999999999999e-06 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384904+0000", 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9627999999999999e-05 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384918+0000", 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4457000000000001e-05 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.384989+0000", 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0133000000000001e-05 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385098+0000", 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010958700000000001 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385447+0000", 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034884299999999999 2024-09-16T14:07:12.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:22 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.385683+0000", 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.105926857, 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00049028899999999998, 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385683+0000", 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385683+0000", 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385685+0000", 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8470000000000001e-06 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385693+0000", 2024-09-16T14:07:12.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5129999999999996e-06 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385717+0000", 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3286e-05 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385726+0000", 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1200000000000008e-06 2024-09-16T14:07:12.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385808+0000", 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2025000000000002e-05 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.385942+0000", 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013404000000000001 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.386173+0000", 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023145800000000001 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:24 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.387231+0000", 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.104378917, 2024-09-16T14:07:12.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00049336900000000001, 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 24 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387231+0000", 2024-09-16T14:07:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387231+0000", 2024-09-16T14:07:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387233+0000", 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.956e-06 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387243+0000", 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387269+0000", 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6605999999999999e-05 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387280+0000", 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0223e-05 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387384+0000", 2024-09-16T14:07:12.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104768 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387521+0000", 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013694400000000001 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.387724+0000", 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020303199999999999 2024-09-16T14:07:12.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:25 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [watch watch cookie 140383153665024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.388129+0000", 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1034805430000001, 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002123921, 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 25 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388129+0000", 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388129+0000", 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388131+0000", 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.996e-06 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388140+0000", 2024-09-16T14:07:12.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9330000000000007e-06 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388166+0000", 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999999e-05 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388176+0000", 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0560000000000001e-05 2024-09-16T14:07:12.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388272+0000", 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6200999999999996e-05 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388417+0000", 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014444300000000001 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.388668+0000", 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025128800000000001 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389878+0000", 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012102650000000001 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.390008+0000", 2024-09-16T14:07:12.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012926300000000001 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.390099+0000", 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1218000000000007e-05 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.390253+0000", 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000154554 2024-09-16T14:07:12.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4406.0:25 3.e e26/23 ondisk, result = 0)", 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.389871+0000", 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1017392850000001, 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039019100000000002, 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389871+0000", 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389871+0000", 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389872+0000", 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389879+0000", 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7719999999999997e-06 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389891+0000", 2024-09-16T14:07:12.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2351000000000001e-05 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389898+0000", 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3139999999999998e-06 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389974+0000", 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5918000000000001e-05 2024-09-16T14:07:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.389994+0000", 2024-09-16T14:07:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9307999999999999e-05 2024-09-16T14:07:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.390261+0000", 2024-09-16T14:07:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000267314 2024-09-16T14:07:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.569 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:12.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:12.571 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:07:12.572 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:12.562+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:12.666 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:12.666 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:12.666 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:12.666 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:12.671 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:12.666+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:12.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:12.772 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:12.766+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:12.818 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:12.818 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:12.818 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:12.818 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:12.818 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:12.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:12.873 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:12.866+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:12.923 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:12.923 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:12.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:12.923 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:12.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:12.923 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.442404206999999, 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.095465191000000005, 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479100+0000", 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9840000000000001e-06 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479107+0000", 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7920000000000004e-06 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479120+0000", 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3195e-05 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479131+0000", 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1416999999999999e-05 2024-09-16T14:07:12.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479210+0000", 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8223e-05 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479224+0000", 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.436e-05 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570355+0000", 2024-09-16T14:07:12.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091131223999999997 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570668+0000", 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031290000000000002 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570990+0000", 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032147600000000001 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574367+0000", 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033769680000000002 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574482+0000", 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115021 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574528+0000", 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6035999999999998e-05 2024-09-16T14:07:12.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574560+0000", 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2595000000000002e-05 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4241.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.574323+0000", 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.347175976000001, 2024-09-16T14:07:12.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024580499999999998, 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574323+0000", 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574323+0000", 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574325+0000", 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.263e-06 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574337+0000", 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2305e-05 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574356+0000", 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9602999999999998e-05 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574364+0000", 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9950000000000005e-06 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574469+0000", 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001048 2024-09-16T14:07:12.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574476+0000", 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7159999999999997e-06 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574569+0000", 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3122999999999996e-05 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.577942+0000", 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.343557685, 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027915200000000001, 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577942+0000", 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577942+0000", 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577944+0000", 2024-09-16T14:07:12.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0559999999999999e-06 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577964+0000", 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0254999999999998e-05 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577986+0000", 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2560000000000001e-05 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577997+0000", 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0939e-05 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578072+0000", 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5048000000000004e-05 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578221+0000", 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014829399999999999 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:12.939 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.578428+0000", 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.343070927999999, 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024553800000000001, 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578428+0000", 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578428+0000", 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578430+0000", 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.378e-06 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578437+0000", 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9580000000000001e-06 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578454+0000", 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7311000000000001e-05 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578462+0000", 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6760000000000007e-06 2024-09-16T14:07:12.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578527+0000", 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5106999999999999e-05 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578674+0000", 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014710799999999999 2024-09-16T14:07:12.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.944 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.944 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T14:07:12.944 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:01.489554+0000", 2024-09-16T14:07:12.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.431945312, 2024-09-16T14:07:12.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058690039999999997, 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489554+0000", 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489554+0000", 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489558+0000", 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.642e-06 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489573+0000", 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5569e-05 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489613+0000", 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9746999999999997e-05 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489634+0000", 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1577e-05 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489731+0000", 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6243000000000007e-05 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489738+0000", 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6979999999999998e-06 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.492405+0000", 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026663500000000001 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.492545+0000", 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000140393 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.492688+0000", 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014267300000000001 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.493926+0000", 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001238128 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495338+0000", 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001412176 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495386+0000", 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8024e-05 2024-09-16T14:07:12.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495423+0000", 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6783999999999999e-05 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4241.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:01.495204+0000", 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.426295724999999, 2024-09-16T14:07:12.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022990099999999999, 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495204+0000", 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495204+0000", 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495205+0000", 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.438e-06 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495212+0000", 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3549999999999999e-06 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495232+0000", 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9922e-05 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495255+0000", 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2694999999999999e-05 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495324+0000", 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8963999999999999e-05 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495331+0000", 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4800000000000004e-06 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495433+0000", 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010204699999999999 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.955 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:02.499179+0000", 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.422319915999999, 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060043609999999997, 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.956 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499179+0000", 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499179+0000", 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499182+0000", 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0400000000000001e-06 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499199+0000", 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7056000000000001e-05 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499238+0000", 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8584999999999997e-05 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499254+0000", 2024-09-16T14:07:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-16T14:07:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499351+0000", 2024-09-16T14:07:12.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7764999999999997e-05 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499359+0000", 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9519999999999994e-06 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.502323+0000", 2024-09-16T14:07:12.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002963959 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.502439+0000", 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011597199999999999 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.502645+0000", 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020520900000000001 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504591+0000", 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001946487 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504981+0000", 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038973600000000001 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.505096+0000", 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115707 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.505184+0000", 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7273000000000003e-05 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4241.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:02.504838+0000", 2024-09-16T14:07:12.963 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.416661587, 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035749099999999998, 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504838+0000", 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504838+0000", 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504839+0000", 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2780000000000001e-06 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504846+0000", 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7050000000000002e-06 2024-09-16T14:07:12.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504862+0000", 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6507999999999998e-05 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504870+0000", 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6699999999999994e-06 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504941+0000", 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1258000000000001e-05 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504962+0000", 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:12.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.505195+0000", 2024-09-16T14:07:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000232883 2024-09-16T14:07:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:12.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:12.968 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:12.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:13.016 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:13.010+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:13.075 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:13.070+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:13.078 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:13.078 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:13.078 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:13.078 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:13.087 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:13.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:13.176 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:13.170+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:13.184 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:13.184 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:13.185 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:13.185 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:13.185 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:13.193 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:13.277 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:13.270+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:13.289 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:13 3.8 3:14a6dda4:::rbd_object_map.113631a28a63:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:11.363695+0000", 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.923917849, 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085796550000000003, 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363695+0000", 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363695+0000", 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363697+0000", 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3369999999999998e-06 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363711+0000", 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.332e-05 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363731+0000", 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9927000000000001e-05 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363742+0000", 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1493e-05 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363884+0000", 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014240300000000001 2024-09-16T14:07:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363894+0000", 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9799999999999993e-06 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.367245+0000", 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033506009999999999 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.367444+0000", 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019903 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.367715+0000", 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000270802 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.369355+0000", 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016402960000000001 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372145+0000", 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027895369999999999 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372223+0000", 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8517999999999996e-05 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372275+0000", 2024-09-16T14:07:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1411e-05 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(client.4406.0:13 3.8 e25/23 ondisk, result = 0)", 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:11.371971+0000", 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.9156419979999999, 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031949799999999998, 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.371971+0000", 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.371971+0000", 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.371972+0000", 2024-09-16T14:07:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.637e-06 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.371985+0000", 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2391999999999999e-05 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372010+0000", 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5185e-05 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372024+0000", 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4371e-05 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372122+0000", 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7219000000000005e-05 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372131+0000", 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5740000000000002e-06 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372290+0000", 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015912000000000001 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:13.301 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:13.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:13.303 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:13.323 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:13.343 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:13.362 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:13.378 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:13.374+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:13.382 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:13.405 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:13.406 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:13.406 INFO:teuthology.orchestra.run.smithi079.stdout:skipping zero size read 2024-09-16T14:07:13.406 INFO:teuthology.orchestra.run.smithi079.stdout:skipping zero size read 2024-09-16T14:07:13.412 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:13.412 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:13.416 INFO:teuthology.orchestra.run.smithi079.stdout:6 write 0xa3f7f6 thru 0xa40da4 (0x15af bytes) 2024-09-16T14:07:13.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:7 read 0xfa65f thru 0xff541 (0x4ee3 bytes) 2024-09-16T14:07:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:8 read 0x314e3a thru 0x31897e (0x3b45 bytes) 2024-09-16T14:07:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:9 punch from 0x4a39c7 to 0x4a58a3, (0x1edc bytes) 2024-09-16T14:07:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:10 read 0x4af80d thru 0x4b2ada (0x32ce bytes) 2024-09-16T14:07:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:12 read 0x805d thru 0x1430b (0xc2af bytes) 2024-09-16T14:07:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:13 read 0x5e615 thru 0x6e465 (0xfe51 bytes) 2024-09-16T14:07:13.426 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:13.445 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:13.446 INFO:teuthology.orchestra.run.smithi079.stdout:15 write 0x9580c65 thru 0x958d6ce (0xca6a bytes) 2024-09-16T14:07:13.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:13.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:13.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:13.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:13.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:13.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:13.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:13.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:13.453 INFO:teuthology.orchestra.run.smithi079.stdout:truncating to largest ever: 0xd7b558e 2024-09-16T14:07:13.460 INFO:teuthology.orchestra.run.smithi079.stdout:17 trunc from 0x958d6cf to 0xd7b558e 2024-09-16T14:07:13.470 INFO:teuthology.orchestra.run.smithi079.stdout:19 writesame 0x4ce7c6c thru 0x4cf5849 data_size 0x9f(0xdbde bytes) 2024-09-16T14:07:13.479 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:13.474+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:13.493 INFO:teuthology.orchestra.run.smithi079.stdout:20 trunc from 0xd7b558e to 0x2e2b3e9 2024-09-16T14:07:13.512 INFO:teuthology.orchestra.run.smithi079.stdout:21 punch from 0x178d046 to 0x1798a09, (0xb9c3 bytes) 2024-09-16T14:07:13.512 INFO:teuthology.orchestra.run.smithi079.stdout:23 read 0x2308f58 thru 0x23108f0 (0x7999 bytes) 2024-09-16T14:07:13.513 INFO:teuthology.orchestra.run.smithi079.stdout:25 read 0x2c8feac thru 0x2c9b79f (0xb8f4 bytes) 2024-09-16T14:07:13.513 INFO:teuthology.orchestra.run.smithi079.stdout:27 read 0x1e02d9b thru 0x1e05097 (0x22fd bytes) 2024-09-16T14:07:13.522 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:13.522 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:13.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:13.534 INFO:teuthology.orchestra.run.smithi079.stdout:28 write 0x97d0e93 thru 0x97d9349 (0x84b7 bytes) 2024-09-16T14:07:13.540 INFO:teuthology.orchestra.run.smithi079.stdout:30 read 0x3205d0a thru 0x3214fe3 (0xf2da bytes) 2024-09-16T14:07:13.541 INFO:teuthology.orchestra.run.smithi079.stdout:32 write 0xf0ea2f thru 0xf1a0b4 (0xb686 bytes) 2024-09-16T14:07:13.548 INFO:teuthology.orchestra.run.smithi079.stdout:34 writesame 0x1cb15e1 thru 0x1cbfa3e data_size 0x1(0xe45e bytes) 2024-09-16T14:07:13.580 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:13.574+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:13.597 INFO:teuthology.orchestra.run.smithi079.stdout:35 write 0x9f39b86 thru 0x9f440d2 (0xa54d bytes) 2024-09-16T14:07:13.603 INFO:teuthology.orchestra.run.smithi079.stdout:36 writesame 0x5800ad0 thru 0x580dbce data_size 0x1(0xd0ff bytes) 2024-09-16T14:07:13.645 INFO:teuthology.orchestra.run.smithi079.stdout:37 punch from 0x70e8c2c to 0x70f2595, (0x9969 bytes) 2024-09-16T14:07:13.645 INFO:teuthology.orchestra.run.smithi079.stdout:38 read 0x2d75794 thru 0x2d78c64 (0x34d1 bytes) 2024-09-16T14:07:13.645 INFO:teuthology.orchestra.run.smithi079.stdout:39 clone 1 order 19 su 65536 sc 6 2024-09-16T14:07:13.671 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:13.672 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:13.680 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:13.681 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:13.674+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:13.776 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:13.776 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:13.782 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:13.778+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:13.785 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:13.882 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:13.882 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:13.883 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:13.874+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:13.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:13.984 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:13.978+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:13.987 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:13.988 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:13.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:14.085 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:14.078+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:14.141 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:14.141 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:14.150 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:14.195 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:14.190+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:14.256 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:14.256 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:14.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:14.286 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:14.278+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:14.361 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:14.361 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:14.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:14.416 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:14.410+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:14.484 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:14.484 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:14.488 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:14.482+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:14.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:14.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:07:14.589 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:14.582+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:14.612 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:14.613 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:14.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:14.690 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:14.686+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:14.723 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:14.723 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:14.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:14.791 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:14.786+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:14.795 INFO:teuthology.orchestra.run.smithi079.stdout:40 trunc from 0x9f440d3 to 0x8de46c 2024-09-16T14:07:14.825 INFO:teuthology.orchestra.run.smithi079.stdout:41 read 0x238a33 thru 0x23e8c4 (0x5e92 bytes) 2024-09-16T14:07:14.831 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:14.832 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:14.843 INFO:teuthology.orchestra.run.smithi079.stdout:42 trunc from 0x8de46c to 0x9ba373a 2024-09-16T14:07:14.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:14.857 INFO:teuthology.orchestra.run.smithi079.stdout:43 punch from 0x2011d62 to 0x201fe22, (0xe0c0 bytes) 2024-09-16T14:07:14.857 INFO:teuthology.orchestra.run.smithi079.stdout:45 read 0x8c1789f thru 0x8c2171d (0x9e7f bytes) 2024-09-16T14:07:14.857 INFO:teuthology.orchestra.run.smithi079.stdout:47 write 0x7c1bf59 thru 0x7c2a4de (0xe586 bytes) 2024-09-16T14:07:14.868 INFO:teuthology.orchestra.run.smithi079.stdout:49 read 0x1975016 thru 0x1980137 (0xb122 bytes) 2024-09-16T14:07:14.868 INFO:teuthology.orchestra.run.smithi079.stdout:51 read 0x8435292 thru 0x8440b5d (0xb8cc bytes) 2024-09-16T14:07:14.869 INFO:teuthology.orchestra.run.smithi079.stdout:52 read 0x3f265b6 thru 0x3f2aad8 (0x4523 bytes) 2024-09-16T14:07:14.869 INFO:teuthology.orchestra.run.smithi079.stdout:53 write 0x3c390ce thru 0x3c40758 (0x768b bytes) 2024-09-16T14:07:14.874 INFO:teuthology.orchestra.run.smithi079.stdout:54 read 0x86f2694 thru 0x86f600e (0x397b bytes) 2024-09-16T14:07:14.885 INFO:teuthology.orchestra.run.smithi079.stdout:55 trunc from 0x9ba373a to 0x320e275 2024-09-16T14:07:14.887 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:07:14.887 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:07:14.888 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T14:07:14.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T14:07:14.910 INFO:teuthology.orchestra.run.smithi079.stdout:56 trunc from 0x320e275 to 0x5be24b2 2024-09-16T14:07:14.910 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:14.906+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:14.928 INFO:teuthology.orchestra.run.smithi079.stdout:57 trunc from 0x5be24b2 to 0x247848c 2024-09-16T14:07:14.939 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:14.957 INFO:teuthology.orchestra.run.smithi079.stdout:59 write 0x57e0b81 thru 0x57ebfcc (0xb44c bytes) 2024-09-16T14:07:14.964 INFO:teuthology.orchestra.run.smithi079.stdout:62 punch from 0x6a92cf to 0x6aa2d3, (0x1004 bytes) 2024-09-16T14:07:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:63 trunc from 0x57ebfcd to 0x507b9a8 2024-09-16T14:07:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:14.966 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:14.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:14.986 INFO:teuthology.orchestra.run.smithi079.stdout:64 read 0x4d189b3 thru 0x4d26784 (0xddd2 bytes) 2024-09-16T14:07:15.003 INFO:teuthology.orchestra.run.smithi079.stdout:65 write 0x74d6160 thru 0x74e5ce9 (0xfb8a bytes) 2024-09-16T14:07:15.003 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:14.998+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:15.013 INFO:teuthology.orchestra.run.smithi079.stdout:66 read 0x3f01816 thru 0x3f08397 (0x6b82 bytes) 2024-09-16T14:07:15.013 INFO:teuthology.orchestra.run.smithi079.stdout:67 read 0x7a1501 thru 0x7a26c1 (0x11c1 bytes) 2024-09-16T14:07:15.013 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.030 INFO:teuthology.orchestra.run.smithi079.stdout:69 writesame 0x91cd1d8 thru 0x91ce5b9 data_size 0x5(0x13e2 bytes) 2024-09-16T14:07:15.030 INFO:teuthology.orchestra.run.smithi079.stdout:70 read 0x820d24c thru 0x821318b (0x5f40 bytes) 2024-09-16T14:07:15.030 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.030 INFO:teuthology.orchestra.run.smithi079.stdout:72 read 0x60716bd thru 0x607b7ea (0xa12e bytes) 2024-09-16T14:07:15.030 INFO:teuthology.orchestra.run.smithi079.stdout:73 read 0x88f475a thru 0x890433b (0xfbe2 bytes) 2024-09-16T14:07:15.043 INFO:teuthology.orchestra.run.smithi079.stdout:76 trunc from 0x91ce5ba to 0x19b1b8f 2024-09-16T14:07:15.067 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.067 INFO:teuthology.orchestra.run.smithi079.stdout:81 trunc from 0x19b1b8f to 0x1ca7397 2024-09-16T14:07:15.079 INFO:teuthology.orchestra.run.smithi079.stdout:82 read 0xdbcaa7 thru 0xdbd84c (0xda6 bytes) 2024-09-16T14:07:15.079 INFO:teuthology.orchestra.run.smithi079.stdout:83 punch from 0x17a8275 to 0x17b48d0, (0xc65b bytes) 2024-09-16T14:07:15.079 INFO:teuthology.orchestra.run.smithi079.stdout:84 read 0xd17f3f thru 0xd207c2 (0x8884 bytes) 2024-09-16T14:07:15.084 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:15.084 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:15.092 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:15.086+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:15.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:15.104 INFO:teuthology.orchestra.run.smithi079.stdout:85 write 0x7ba4b41 thru 0x7ba6bb1 (0x2071 bytes) 2024-09-16T14:07:15.115 INFO:teuthology.orchestra.run.smithi079.stdout:86 trunc from 0x7ba6bb2 to 0x4a908f5 2024-09-16T14:07:15.164 INFO:teuthology.orchestra.run.smithi079.stdout:87 write 0xe92b942 thru 0xe9367d6 (0xae95 bytes) 2024-09-16T14:07:15.173 INFO:teuthology.orchestra.run.smithi079.stdout:88 read 0x137781d thru 0x137a3b7 (0x2b9b bytes) 2024-09-16T14:07:15.173 INFO:teuthology.orchestra.run.smithi079.stdout:89 write 0x9db2f74 thru 0x9dbf298 (0xc325 bytes) 2024-09-16T14:07:15.177 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 28, 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "a22b38a6-94db-470d-bf35-45d7605f3bb0", 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-16T14:06:40.573726+0000", 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-16T14:07:14.550745+0000", 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-16T14:06:51.260361+0000", 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-16T14:06:42.748450+0000", 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-16T14:07:15.178 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 5, 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-16T14:07:15.179 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 3, 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "jewel", 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "quincy", 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:52.822826+0000", 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:07:15.180 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-16T14:07:15.181 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "18", 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:07:15.182 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:07:15.183 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:07:15.184 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:07:15.185 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:58.526935+0000", 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:07:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:07:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "22", 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:07:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 22, 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:07:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:07:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:07:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:07:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:07:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:07:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:07:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:07:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:07:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:07:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "pool_client.0", 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:07:08.375919+0000", 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:07:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-16T14:07:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "28", 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:07:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 4, 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 28, 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:07:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:07:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:07:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a060b161-fd5a-4cfb-bac8-f2acc32d7955", 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-16T14:07:15.199 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.200 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:15.202 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/22196", 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/22196", 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6824/22196", 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/22196", 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:15.204 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "f5ba4b39-ceb8-46b3-938e-57249ed84b87", 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:15.205 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-16T14:07:15.209 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/22195", 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/22195", 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/22195", 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/22195", 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:15.210 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "8b657f92-66c9-46a3-a31f-0bd0be414e4e", 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:15.211 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.212 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:15.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-16T14:07:15.214 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:15.215 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-16T14:07:15.216 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6825/22197", 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6827/22197", 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6831/22197", 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6829/22197", 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "15937bd7-8964-4140-9f86-b3093c318d1e", 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:15.218 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.219 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-16T14:07:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-16T14:07:15.222 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/22193", 2024-09-16T14:07:15.223 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/22193", 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/22193", 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/22193", 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.224 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "714baf80-ade0-4e88-ad4e-5171add1fc6a", 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 14, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:15.225 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6808", 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.226 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6809", 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6810", 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:15.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6811", 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6814", 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6815", 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.229 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6812", 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6813", 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:15.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6809/22362", 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6811/22362", 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6815/22362", 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6813/22362", 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:07:15.231 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e5a21b84-81ab-45da-a081-3b504f5c4f3d", 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6800", 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6801", 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6802", 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6803", 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6806", 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6807", 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:15.235 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6804", 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6805", 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.236 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6801/22363", 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6803/22363", 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6807/22363", 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6805/22363", 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a009857f-f798-4154-9ff4-f524f60b6c61", 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:15.237 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6824", 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:15.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6825", 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6826", 2024-09-16T14:07:15.239 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6827", 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6830", 2024-09-16T14:07:15.240 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6831", 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6828", 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6829", 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6825/22364", 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6827/22364", 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6831/22364", 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6829/22364", 2024-09-16T14:07:15.242 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e3506002-ba8b-4251-8ef1-136334574a6d", 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6816", 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.244 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6817", 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6818", 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.245 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6819", 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6822", 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6823", 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6820", 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6821", 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6817/22365", 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6819/22365", 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6823/22365", 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6821/22365", 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:15.248 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.104631+0000", 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987403+0000", 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:15.250 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.018134+0000", 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987645+0000", 2024-09-16T14:07:15.251 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.721507+0000", 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.252 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.869796+0000", 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:15.253 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.206576+0000", 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.038098+0000", 2024-09-16T14:07:15.254 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-16T14:07:15.255 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [ 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: "snaps": [ 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: "begin": 2, 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: "length": 1 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.256 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.257 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:15.259 INFO:teuthology.orchestra.run.smithi079.stdout:90 write 0xaf65315 thru 0xaf6afa0 (0x5c8c bytes) 2024-09-16T14:07:15.259 INFO:teuthology.orchestra.run.smithi079.stdout:91 read 0x56b7cb3 thru 0x56bad1d (0x306b bytes) 2024-09-16T14:07:15.259 INFO:teuthology.orchestra.run.smithi079.stdout:92 punch from 0x7270c14 to 0x72733a9, (0x2795 bytes) 2024-09-16T14:07:15.259 INFO:teuthology.orchestra.run.smithi079.stdout:93 read 0xc53a74d thru 0xc543381 (0x8c35 bytes) 2024-09-16T14:07:15.259 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.259 INFO:teuthology.orchestra.run.smithi079.stdout:95 write 0x9a9f872 thru 0x9aa591f (0x60ae bytes) 2024-09-16T14:07:15.260 INFO:tasks.thrashosds.thrasher:j is {'epoch': 28, 'fsid': 'a22b38a6-94db-470d-bf35-45d7605f3bb0', 'created': '2024-09-16T14:06:40.573726+0000', 'modified': '2024-09-16T14:07:14.550745+0000', 'last_up_change': '2024-09-16T14:06:51.260361+0000', 'last_in_change': '2024-09-16T14:06:42.748450+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T14:06:52.822826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T14:06:58.526935+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T14:07:08.375919+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': 'a060b161-fd5a-4cfb-bac8-f2acc32d7955', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 22196}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 22196}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6824', 'nonce': 22196}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 22196}]}, 'public_addr': '172.21.15.79:6817/22196', 'cluster_addr': '172.21.15.79:6819/22196', 'heartbeat_back_addr': '172.21.15.79:6824/22196', 'heartbeat_front_addr': '172.21.15.79:6821/22196', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f5ba4b39-ceb8-46b3-938e-57249ed84b87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 22195}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 22195}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 22195}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 22195}]}, 'public_addr': '172.21.15.79:6809/22195', 'cluster_addr': '172.21.15.79:6811/22195', 'heartbeat_back_addr': '172.21.15.79:6815/22195', 'heartbeat_front_addr': '172.21.15.79:6813/22195', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8b657f92-66c9-46a3-a31f-0bd0be414e4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6823', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 22197}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 22197}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 22197}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 22197}]}, 'public_addr': '172.21.15.79:6825/22197', 'cluster_addr': '172.21.15.79:6827/22197', 'heartbeat_back_addr': '172.21.15.79:6831/22197', 'heartbeat_front_addr': '172.21.15.79:6829/22197', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '15937bd7-8964-4140-9f86-b3093c318d1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 22193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 22193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 22193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 22193}]}, 'public_addr': '172.21.15.79:6801/22193', 'cluster_addr': '172.21.15.79:6803/22193', 'heartbeat_back_addr': '172.21.15.79:6807/22193', 'heartbeat_front_addr': '172.21.15.79:6805/22193', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '714baf80-ade0-4e88-ad4e-5171add1fc6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 22362}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 22362}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 22362}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 22362}]}, 'public_addr': '172.21.15.164:6809/22362', 'cluster_addr': '172.21.15.164:6811/22362', 'heartbeat_back_addr': '172.21.15.164:6815/22362', 'heartbeat_front_addr': '172.21.15.164:6813/22362', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e5a21b84-81ab-45da-a081-3b504f5c4f3d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 22363}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 22363}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 22363}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 22363}]}, 'public_addr': '172.21.15.164:6801/22363', 'cluster_addr': '172.21.15.164:6803/22363', 'heartbeat_back_addr': '172.21.15.164:6807/22363', 'heartbeat_front_addr': '172.21.15.164:6805/22363', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a009857f-f798-4154-9ff4-f524f60b6c61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 22364}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 22364}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 22364}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 22364}]}, 'public_addr': '172.21.15.164:6825/22364', 'cluster_addr': '172.21.15.164:6827/22364', 'heartbeat_back_addr': '172.21.15.164:6831/22364', 'heartbeat_front_addr': '172.21.15.164:6829/22364', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e3506002-ba8b-4251-8ef1-136334574a6d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 22365}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 22365}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 22365}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 22365}]}, 'public_addr': '172.21.15.164:6817/22365', 'cluster_addr': '172.21.15.164:6819/22365', 'heartbeat_back_addr': '172.21.15.164:6823/22365', 'heartbeat_front_addr': '172.21.15.164:6821/22365', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.104631+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987403+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.018134+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987645+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.721507+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.869796+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.206576+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.038098+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T14:07:15.261 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T14:07:15.261 INFO:teuthology.orchestra.run.smithi079.stdout:96 writesame 0xcb8df6e thru 0xcb92323 data_size 0x3(0x43b6 bytes) 2024-09-16T14:07:15.261 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:15.261 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:15.261 INFO:teuthology.orchestra.run.smithi079.stdout:97 writesame 0x1a6f5e4 thru 0x1a732bd data_size 0x1(0x3cda bytes) 2024-09-16T14:07:15.262 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:15.263 INFO:teuthology.orchestra.run.smithi079.stdout:98 writesame 0x591e67b thru 0x5925a3b data_size 0x1(0x73c1 bytes) 2024-09-16T14:07:15.263 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.263 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.263 INFO:teuthology.orchestra.run.smithi079.stdout:103 read 0xa918245 thru 0xa927763 (0xf51f bytes) 2024-09-16T14:07:15.263 INFO:teuthology.orchestra.run.smithi079.stdout:104 write 0xe7b014b thru 0xe7bed54 (0xec0a bytes) 2024-09-16T14:07:15.263 INFO:teuthology.orchestra.run.smithi079.stdout:105 read 0x72aeb2e thru 0x72b76bb (0x8b8e bytes) 2024-09-16T14:07:15.264 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:15.254+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:15.264 INFO:teuthology.orchestra.run.smithi079.stdout:106 trunc from 0xe9367d7 to 0x5f52435 2024-09-16T14:07:15.284 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.284 INFO:teuthology.orchestra.run.smithi079.stdout:108 write 0x30ff7e8 thru 0x310f25c (0xfa75 bytes) 2024-09-16T14:07:15.294 INFO:teuthology.orchestra.run.smithi079.stdout:109 punch from 0x5eacfec to 0x5eb4fb3, (0x7fc7 bytes) 2024-09-16T14:07:15.294 INFO:teuthology.orchestra.run.smithi079.stdout:111 read 0x2139ca5 thru 0x21411ac (0x7508 bytes) 2024-09-16T14:07:15.294 INFO:teuthology.orchestra.run.smithi079.stdout:112 read 0x273caa7 thru 0x274baba (0xf014 bytes) 2024-09-16T14:07:15.294 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.309 INFO:teuthology.orchestra.run.smithi079.stdout:115 write 0x71ad3ad thru 0x71b5137 (0x7d8b bytes) 2024-09-16T14:07:15.316 INFO:teuthology.orchestra.run.smithi079.stdout:117 punch from 0x5be732e to 0x5bf6227, (0xeef9 bytes) 2024-09-16T14:07:15.316 INFO:teuthology.orchestra.run.smithi079.stdout:119 punch from 0x364b8fa to 0x36594c4, (0xdbca bytes) 2024-09-16T14:07:15.321 INFO:teuthology.orchestra.run.smithi079.stdout:120 trunc from 0x71b5138 to 0x54ef8b0 2024-09-16T14:07:15.346 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.346 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.346 INFO:teuthology.orchestra.run.smithi079.stdout:124 punch from 0x2738be0 to 0x2738ff0, (0x410 bytes) 2024-09-16T14:07:15.346 INFO:teuthology.orchestra.run.smithi079.stdout:125 read 0x52e7566 thru 0x52ea11b (0x2bb6 bytes) 2024-09-16T14:07:15.347 INFO:teuthology.orchestra.run.smithi079.stdout:126 trunc from 0x54ef8b0 to 0x4bca352 2024-09-16T14:07:15.359 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:15.354+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:15.368 INFO:teuthology.orchestra.run.smithi079.stdout:128 trunc from 0x4bca352 to 0x891d23f 2024-09-16T14:07:15.381 INFO:teuthology.orchestra.run.smithi079.stdout:129 writesame 0x453e5fc thru 0x453f1bf data_size 0x6(0xbc4 bytes) 2024-09-16T14:07:15.388 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.388 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.392 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:15.392 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:15.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:15.405 INFO:teuthology.orchestra.run.smithi079.stdout:132 writesame 0x9e6f67b thru 0x9e729b4 data_size 0x2(0x333a bytes) 2024-09-16T14:07:15.420 INFO:teuthology.orchestra.run.smithi079.stdout:133 write 0xa84baae thru 0xa854023 (0x8576 bytes) 2024-09-16T14:07:15.430 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.430 INFO:teuthology.orchestra.run.smithi079.stdout:136 read 0x79a0eda thru 0x79a2b42 (0x1c69 bytes) 2024-09-16T14:07:15.445 INFO:teuthology.orchestra.run.smithi079.stdout:137 trunc from 0xa854024 to 0x296c0d1 2024-09-16T14:07:15.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:15.459 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:15.454+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:15.480 INFO:teuthology.orchestra.run.smithi079.stdout:138 trunc from 0x296c0d1 to 0x77fe368 2024-09-16T14:07:15.506 INFO:teuthology.orchestra.run.smithi079.stdout:139 write 0x87b4fcb thru 0x87ba4d6 (0x550c bytes) 2024-09-16T14:07:15.522 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:15.523 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:15.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:15.537 INFO:teuthology.orchestra.run.smithi079.stdout:140 write 0xc1763fa thru 0xc186295 (0xfe9c bytes) 2024-09-16T14:07:15.544 INFO:teuthology.orchestra.run.smithi079.stdout:141 write 0x8466293 thru 0x846f8df (0x964d bytes) 2024-09-16T14:07:15.561 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:15.554+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:15.563 INFO:teuthology.orchestra.run.smithi079.stdout:143 write 0xd8cdaed thru 0xd8d4f14 (0x7428 bytes) 2024-09-16T14:07:15.578 INFO:teuthology.orchestra.run.smithi079.stdout:146 read 0x1e1b48f thru 0x1e262bd (0xae2f bytes) 2024-09-16T14:07:15.579 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:07:15.595 INFO:teuthology.orchestra.run.smithi079.stdout:149 write 0xeadec93 thru 0xeaeebca (0xff38 bytes) 2024-09-16T14:07:15.603 INFO:teuthology.orchestra.run.smithi079.stdout:151 punch from 0xc62fe43 to 0xc63d11d, (0xd2da bytes) 2024-09-16T14:07:15.603 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.603 INFO:teuthology.orchestra.run.smithi079.stdout:154 read 0x553f931 thru 0x5543d6c (0x443c bytes) 2024-09-16T14:07:15.603 INFO:teuthology.orchestra.run.smithi079.stdout:155 write 0x5edc604 thru 0x5ee340d (0x6e0a bytes) 2024-09-16T14:07:15.607 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:15.610 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.627 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:15.632 INFO:teuthology.orchestra.run.smithi079.stdout:157 trunc from 0xeaeebcb to 0xd64f02 2024-09-16T14:07:15.651 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:15.651 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:15.651 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:15.658 INFO:teuthology.orchestra.run.smithi079.stdout:158 trunc from 0xd64f02 to 0x50faf1e 2024-09-16T14:07:15.661 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:15.654+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:15.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:15.676 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:15.690 INFO:teuthology.orchestra.run.smithi079.stdout:159 writesame 0xaca1adb thru 0xacb143b data_size 0x1(0xf961 bytes) 2024-09-16T14:07:15.691 INFO:teuthology.orchestra.run.smithi079.stdout:161 punch from 0x3903220 to 0x3910ab3, (0xd893 bytes) 2024-09-16T14:07:15.697 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:15.707 INFO:teuthology.orchestra.run.smithi079.stdout:162 write 0xe1c47fb thru 0xe1c6eee (0x26f4 bytes) 2024-09-16T14:07:15.716 INFO:teuthology.orchestra.run.smithi079.stdout:163 write 0xc1db60b thru 0xc1dbe52 (0x848 bytes) 2024-09-16T14:07:15.719 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:15.722 INFO:teuthology.orchestra.run.smithi079.stdout:164 writesame 0x2976c0a thru 0x297a17c data_size 0x1(0x3573 bytes) 2024-09-16T14:07:15.737 INFO:teuthology.orchestra.run.smithi079.stdout:166 read 0x9a9911f thru 0x9aa7fee (0xeed0 bytes) 2024-09-16T14:07:15.737 INFO:teuthology.orchestra.run.smithi079.stdout:167 read 0x67e0a70 thru 0x67ed1ca (0xc75b bytes) 2024-09-16T14:07:15.737 INFO:teuthology.orchestra.run.smithi079.stdout:168 read 0x54e82a thru 0x55506f (0x6846 bytes) 2024-09-16T14:07:15.737 INFO:teuthology.orchestra.run.smithi079.stdout:171 punch from 0x52c53f4 to 0x52d5231, (0xfe3d bytes) 2024-09-16T14:07:15.738 INFO:teuthology.orchestra.run.smithi079.stdout:173 read 0xade9958 thru 0xadee18c (0x4835 bytes) 2024-09-16T14:07:15.738 INFO:teuthology.orchestra.run.smithi079.stdout:174 write 0xb3699e9 thru 0xb377c6f (0xe287 bytes) 2024-09-16T14:07:15.746 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:15.750 INFO:teuthology.orchestra.run.smithi079.stdout:175 punch from 0x1f2dc80 to 0x1f39972, (0xbcf2 bytes) 2024-09-16T14:07:15.751 INFO:teuthology.orchestra.run.smithi079.stdout:177 punch from 0x587e6c9 to 0x5885770, (0x70a7 bytes) 2024-09-16T14:07:15.751 INFO:teuthology.orchestra.run.smithi079.stdout:178 write 0xb62f28a thru 0xb63116c (0x1ee3 bytes) 2024-09-16T14:07:15.758 INFO:teuthology.orchestra.run.smithi079.stdout:180 read 0xd0d8117 thru 0xd0dd674 (0x555e bytes) 2024-09-16T14:07:15.758 INFO:teuthology.orchestra.run.smithi079.stdout:181 punch from 0xb26bd82 to 0xb2782a8, (0xc526 bytes) 2024-09-16T14:07:15.762 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:15.754+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:15.771 INFO:teuthology.orchestra.run.smithi079.stdout:184 trunc from 0xe1c6eef to 0x6340b87 2024-09-16T14:07:15.776 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:15.778 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:15.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:15.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:15.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:15.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:15.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:15.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:15.779 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:15.783 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:15.783 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:15.783 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:15.784 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:15.784 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:15.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:15.818 INFO:teuthology.orchestra.run.smithi079.stdout:185 writesame 0xd2db0b9 thru 0xd2ea837 data_size 0x1(0xf77f bytes) 2024-09-16T14:07:15.828 INFO:teuthology.orchestra.run.smithi079.stdout:186 trunc from 0xd2ea838 to 0x7bec3ff 2024-09-16T14:07:15.841 INFO:teuthology.orchestra.run.smithi079.stdout:187 trunc from 0x7bec3ff to 0x783288c 2024-09-16T14:07:15.853 INFO:teuthology.orchestra.run.smithi079.stdout:188 read 0x76d5c4c thru 0x76de2be (0x8673 bytes) 2024-09-16T14:07:15.853 INFO:teuthology.orchestra.run.smithi079.stdout:189 write 0x310a54f thru 0x3119f4c (0xf9fe bytes) 2024-09-16T14:07:15.863 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:15.854+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:15.888 INFO:teuthology.orchestra.run.smithi079.stdout:190 write 0xcfb5762 thru 0xcfb7899 (0x2138 bytes) 2024-09-16T14:07:15.895 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:15.899 INFO:teuthology.orchestra.run.smithi079.stdout:193 trunc from 0xcfb789a to 0xabff22d 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.440267934, 2024-09-16T14:07:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.095465191000000005, 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479100+0000", 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9840000000000001e-06 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479107+0000", 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7920000000000004e-06 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479120+0000", 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3195e-05 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479131+0000", 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1416999999999999e-05 2024-09-16T14:07:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479210+0000", 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8223e-05 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479224+0000", 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.436e-05 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570355+0000", 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091131223999999997 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570668+0000", 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031290000000000002 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570990+0000", 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032147600000000001 2024-09-16T14:07:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574367+0000", 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033769680000000002 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574482+0000", 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115021 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574528+0000", 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6035999999999998e-05 2024-09-16T14:07:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574560+0000", 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2595000000000002e-05 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4241.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.574323+0000", 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.345039702999999, 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024580499999999998, 2024-09-16T14:07:15.930 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574323+0000", 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574323+0000", 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574325+0000", 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.263e-06 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574337+0000", 2024-09-16T14:07:15.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2305e-05 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574356+0000", 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9602999999999998e-05 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574364+0000", 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9950000000000005e-06 2024-09-16T14:07:15.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574469+0000", 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001048 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574476+0000", 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7159999999999997e-06 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574569+0000", 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3122999999999996e-05 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.577942+0000", 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.341421412000001, 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027915200000000001, 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:07:15.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 2 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577942+0000", 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577942+0000", 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577944+0000", 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0559999999999999e-06 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577964+0000", 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0254999999999998e-05 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577986+0000", 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2560000000000001e-05 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.577997+0000", 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0939e-05 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578072+0000", 2024-09-16T14:07:15.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5048000000000004e-05 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578221+0000", 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014829399999999999 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.578428+0000", 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.340934655, 2024-09-16T14:07:15.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024553800000000001, 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578428+0000", 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578428+0000", 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578430+0000", 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.378e-06 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578437+0000", 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9580000000000001e-06 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578454+0000", 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7311000000000001e-05 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578462+0000", 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6760000000000007e-06 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578527+0000", 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5106999999999999e-05 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.578674+0000", 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014710799999999999 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:01.489554+0000", 2024-09-16T14:07:15.943 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.429809039, 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058690039999999997, 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489554+0000", 2024-09-16T14:07:15.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489554+0000", 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489558+0000", 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.642e-06 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489573+0000", 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5569e-05 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489613+0000", 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9746999999999997e-05 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489634+0000", 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1577e-05 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489731+0000", 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6243000000000007e-05 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.489738+0000", 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6979999999999998e-06 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.492405+0000", 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026663500000000001 2024-09-16T14:07:15.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.492545+0000", 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000140393 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.492688+0000", 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014267300000000001 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.493926+0000", 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001238128 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495338+0000", 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001412176 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495386+0000", 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8024e-05 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495423+0000", 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6783999999999999e-05 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4241.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:01.495204+0000", 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.424159452, 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022990099999999999, 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:15.950 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495204+0000", 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495204+0000", 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495205+0000", 2024-09-16T14:07:15.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.438e-06 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495212+0000", 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3549999999999999e-06 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495232+0000", 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9922e-05 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495255+0000", 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2694999999999999e-05 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495324+0000", 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8963999999999999e-05 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495331+0000", 2024-09-16T14:07:15.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4800000000000004e-06 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:01.495433+0000", 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010204699999999999 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:02.499179+0000", 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.420183643, 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060043609999999997, 2024-09-16T14:07:15.954 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499179+0000", 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499179+0000", 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499182+0000", 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0400000000000001e-06 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499199+0000", 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7056000000000001e-05 2024-09-16T14:07:15.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499238+0000", 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8584999999999997e-05 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499254+0000", 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.562e-05 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499351+0000", 2024-09-16T14:07:15.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7764999999999997e-05 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.499359+0000", 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9519999999999994e-06 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.502323+0000", 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002963959 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.502439+0000", 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011597199999999999 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.502645+0000", 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020520900000000001 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504591+0000", 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001946487 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504981+0000", 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038973600000000001 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.505096+0000", 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115707 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.505184+0000", 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7273000000000003e-05 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.960 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4241.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:02.504838+0000", 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.414525314, 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035749099999999998, 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504838+0000", 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504838+0000", 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504839+0000", 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2780000000000001e-06 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504846+0000", 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7050000000000002e-06 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504862+0000", 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6507999999999998e-05 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504870+0000", 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6699999999999994e-06 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504941+0000", 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1258000000000001e-05 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.504962+0000", 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:02.505195+0000", 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000232883 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.964 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:676 3.a 3:5479229a:::rbd_data.1136e0d3367d.0000000000000185:head [set-alloc-hint object_size 524288 write_size 524288,write 308747~2120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.719416+0000", 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.19994685000000001, 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021186500000000001, 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:15.965 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 676 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719416+0000", 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719416+0000", 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719419+0000", 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2449999999999998e-06 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719436+0000", 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6541999999999998e-05 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719480+0000", 2024-09-16T14:07:15.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4016e-05 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719496+0000", 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6053e-05 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719592+0000", 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6031000000000003e-05 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719688+0000", 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6392999999999997e-05 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.719797+0000", 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108343 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.720798+0000", 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010013260000000001 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721452+0000", 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00065384799999999995 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721494+0000", 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2249000000000002e-05 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721535+0000", 2024-09-16T14:07:15.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0604e-05 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4406.0:676 3.a e29/23 ondisk, result = 0)", 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.721384+0000", 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.19797878499999999, 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015495799999999999, 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721384+0000", 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721384+0000", 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721385+0000", 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2900000000000002e-07 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721391+0000", 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.66e-06 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721402+0000", 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0909e-05 2024-09-16T14:07:15.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721408+0000", 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4180000000000002e-06 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721444+0000", 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5608000000000002e-05 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721448+0000", 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3950000000000003e-06 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.721539+0000", 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1038999999999994e-05 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:696 3.a 3:5479229a:::rbd_data.1136e0d3367d.0000000000000185:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:15.975 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.774317+0000", 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.14504591999999999, 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054074539999999999, 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 696 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774317+0000", 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774317+0000", 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774319+0000", 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9599999999999999e-06 2024-09-16T14:07:15.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774326+0000", 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774348+0000", 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2424000000000001e-05 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774358+0000", 2024-09-16T14:07:15.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8169999999999994e-06 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774503+0000", 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145496 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774699+0000", 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019617199999999999 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.775005+0000", 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000305094 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.775934+0000", 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00092900199999999995 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779602+0000", 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036687040000000001 2024-09-16T14:07:15.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779672+0000", 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9783999999999995e-05 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779725+0000", 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2509000000000002e-05 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4406.0:696 3.a e29/23 ondisk, result = 0)", 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.779491+0000", 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.13987169399999999, 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000240523, 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779491+0000", 2024-09-16T14:07:15.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779491+0000", 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779492+0000", 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.097e-06 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779499+0000", 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1580000000000002e-06 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779516+0000", 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7402999999999999e-05 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779525+0000", 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9279999999999999e-06 2024-09-16T14:07:15.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779587+0000", 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2514999999999994e-05 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779595+0000", 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7309999999999993e-06 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779732+0000", 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013669100000000001 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:738 3.d 3:b59fde62:::rbd_data.1136e0d3367d.0000000000000065:head [set-alloc-hint object_size 524288 write_size 524288,write 131072~40781] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.858724+0000", 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.060639038999999999, 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038208690000000002, 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.986 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 738 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858724+0000", 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858724+0000", 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858727+0000", 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5030000000000002e-06 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858768+0000", 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0114000000000003e-05 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858816+0000", 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8130999999999997e-05 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858833+0000", 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7142999999999998e-05 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858961+0000", 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000127749 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.859241+0000", 2024-09-16T14:07:15.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028013000000000002 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.859477+0000", 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000236417 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.861306+0000", 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001829004 2024-09-16T14:07:15.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862413+0000", 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001106489 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862487+0000", 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4430000000000004e-05 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862545+0000", 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7759e-05 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop_reply(client.4406.0:738 3.d e29/23 ondisk, result = 0)", 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.862268+0000", 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.057094602000000001, 2024-09-16T14:07:15.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033480499999999998, 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862268+0000", 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862268+0000", 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862270+0000", 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5650000000000001e-06 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862276+0000", 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7089999999999999e-06 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862295+0000", 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8842999999999999e-05 2024-09-16T14:07:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862303+0000", 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8040000000000007e-06 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862392+0000", 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9023000000000005e-05 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862403+0000", 2024-09-16T14:07:15.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0868e-05 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.862603+0000", 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019999300000000001 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:15.997 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:15.998 INFO:teuthology.orchestra.run.smithi079.stdout:195 trunc from 0xabff22d to 0x543ded0 2024-09-16T14:07:15.999 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:196 punch from 0xf7c689 to 0xf7eb63, (0x24da bytes) 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:199 writesame 0x8ef6272 thru 0x8ef8368 data_size 0x1d(0x20f7 bytes) 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:201 write 0x1e380b7 thru 0x1e3ebcc (0x6b16 bytes) 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:202 read 0x4fb7a71 thru 0x4fc43b3 (0xc943 bytes) 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:203 read 0x4ad75ad thru 0x4ae737a (0xfdce bytes) 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:205 write 0x6c5debe thru 0x6c61d27 (0x3e6a bytes) 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:206 write 0x46c3b0b thru 0x46d14b8 (0xd9ae bytes) 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:208 write 0x7ce6a3a thru 0x7cf17e7 (0xadae bytes) 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:16.001 INFO:teuthology.orchestra.run.smithi079.stdout:211 clone 2 order 21 su 1048576 sc 15 2024-09-16T14:07:16.002 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:15.990+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:16.101 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:16.094+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:16.105 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:16.106 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:16.106 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:16.106 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:16.115 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:16.202 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:16.198+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:16.217 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:16.217 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:16.217 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:16.217 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:16.217 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:16.227 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:16.303 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:16.294+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:16.328 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:13 3.8 3:14a6dda4:::rbd_object_map.113631a28a63:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:11.363695+0000", 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.9587855320000003, 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085796550000000003, 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363695+0000", 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363695+0000", 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363697+0000", 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3369999999999998e-06 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363711+0000", 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.332e-05 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363731+0000", 2024-09-16T14:07:16.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9927000000000001e-05 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363742+0000", 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1493e-05 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363884+0000", 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014240300000000001 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.363894+0000", 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9799999999999993e-06 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.367245+0000", 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033506009999999999 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.367444+0000", 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019903 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.367715+0000", 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000270802 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.369355+0000", 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016402960000000001 2024-09-16T14:07:16.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372145+0000", 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027895369999999999 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372223+0000", 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8517999999999996e-05 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.372275+0000", 2024-09-16T14:07:16.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1411e-05 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:673 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.706351+0000", 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.61612990899999998, 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052742839999999997, 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 673 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706351+0000", 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706351+0000", 2024-09-16T14:07:16.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706353+0000", 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0949999999999998e-06 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706364+0000", 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1134e-05 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706397+0000", 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3442000000000003e-05 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706409+0000", 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1882000000000001e-05 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706485+0000", 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5544999999999996e-05 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706592+0000", 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106895 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.706997+0000", 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040506599999999999 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.708591+0000", 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594179 2024-09-16T14:07:16.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.711546+0000", 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002955671 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.711593+0000", 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6279000000000002e-05 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.711625+0000", 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2095999999999999e-05 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:679 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.736046+0000", 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.58643494699999998, 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059559690000000002, 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 679 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736046+0000", 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736046+0000", 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736047+0000", 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.838e-06 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736056+0000", 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1109999999999997e-06 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736082+0000", 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5703000000000001e-05 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736092+0000", 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.465e-06 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736190+0000", 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8630999999999998e-05 2024-09-16T14:07:16.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736361+0000", 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000170843 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736819+0000", 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00045819300000000002 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.741876+0000", 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050563550000000002 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.741889+0000", 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3618000000000001e-05 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.741951+0000", 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1970999999999993e-05 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742002+0000", 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0241000000000002e-05 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:680 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.736111+0000", 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.58636932600000002, 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079734309999999996, 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.348 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 680 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736111+0000", 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736111+0000", 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736112+0000", 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4099999999999997e-07 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736119+0000", 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4679999999999997e-06 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736132+0000", 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3416e-05 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736140+0000", 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7999999999999999e-06 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.737589+0000", 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014490149999999999 2024-09-16T14:07:16.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.737897+0000", 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030775900000000001 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.739234+0000", 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013372010000000001 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742154+0000", 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002919905 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.743890+0000", 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001736458 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744004+0000", 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113479 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744085+0000", 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0989000000000007e-05 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:681 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.736225+0000", 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.586254991, 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091861960000000006, 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.354 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 681 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736225+0000", 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736225+0000", 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736227+0000", 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.093e-06 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736233+0000", 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2179999999999996e-06 2024-09-16T14:07:16.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736253+0000", 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0219000000000001e-05 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736262+0000", 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7339999999999997e-06 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.740226+0000", 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039642080000000003 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.740418+0000", 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019215899999999999 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.741072+0000", 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00065419399999999998 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.745337+0000", 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042649849999999998 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.745345+0000", 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1769999999999998e-06 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.745383+0000", 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7134000000000001e-05 2024-09-16T14:07:16.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.745412+0000", 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9074999999999999e-05 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:682 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.736273+0000", 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.58620743200000003, 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010329430000000001, 2024-09-16T14:07:16.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 682 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736273+0000", 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736273+0000", 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736274+0000", 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3799999999999996e-07 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736279+0000", 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2390000000000002e-06 2024-09-16T14:07:16.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736299+0000", 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0211e-05 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.736308+0000", 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.789e-06 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742191+0000", 2024-09-16T14:07:16.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058831609999999996 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742299+0000", 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107509 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.743046+0000", 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074741399999999996 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.745828+0000", 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027815079999999998 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.746365+0000", 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00053688599999999998 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.746514+0000", 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014883400000000001 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.746602+0000", 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8941000000000001e-05 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:683 3.6 3:65cfeb1a:::rbd_data.1136e0d3367d.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.742283+0000", 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.58019782600000003, 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067557620000000002, 2024-09-16T14:07:16.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 683 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742283+0000", 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742283+0000", 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742285+0000", 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2970000000000002e-06 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742301+0000", 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5682e-05 2024-09-16T14:07:16.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742326+0000", 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5843e-05 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.742343+0000", 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6753999999999999e-05 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744201+0000", 2024-09-16T14:07:16.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001858111 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744568+0000", 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036689200000000001 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744826+0000", 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025797999999999999 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.748421+0000", 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035945170000000002 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.748965+0000", 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054407900000000005 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.749007+0000", 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2590999999999999e-05 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.749038+0000", 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1016000000000003e-05 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:684 3.1b 3:de1f822b:::rbd_data.1136e0d3367d.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.744325+0000", 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.57815596199999997, 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041572919999999999, 2024-09-16T14:07:16.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 684 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744325+0000", 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744325+0000", 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744327+0000", 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.463e-06 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744341+0000", 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3534e-05 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744414+0000", 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3783999999999997e-05 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744477+0000", 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2248000000000004e-05 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744543+0000", 2024-09-16T14:07:16.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6451999999999993e-05 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744673+0000", 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012992 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.744812+0000", 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013942699999999999 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.746556+0000", 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017441170000000001 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.748372+0000", 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815959 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.748433+0000", 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0133999999999997e-05 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.748482+0000", 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9254e-05 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:688 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.749488+0000", 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.57299266199999999, 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047464650000000001, 2024-09-16T14:07:16.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 688 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.749488+0000", 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.749488+0000", 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.749489+0000", 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9099999999999991e-07 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.749494+0000", 2024-09-16T14:07:16.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.989e-06 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.749504+0000", 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0699e-05 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.749517+0000", 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.257e-05 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.750582+0000", 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010645870000000001 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.750693+0000", 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111335 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.751091+0000", 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039770000000000002 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.753988+0000", 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002897274 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754122+0000", 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013418600000000001 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754173+0000", 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0402000000000003e-05 2024-09-16T14:07:16.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754234+0000", 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1731999999999998e-05 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:727 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.843015+0000", 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.47946578699999998, 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003349187, 2024-09-16T14:07:16.384 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 727 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.843015+0000", 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.843015+0000", 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.843017+0000", 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9740000000000001e-06 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.843027+0000", 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0407999999999999e-05 2024-09-16T14:07:16.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.843055+0000", 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.836e-05 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.843066+0000", 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0977e-05 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.843177+0000", 2024-09-16T14:07:16.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011068100000000001 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.843422+0000", 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024526399999999997 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.844131+0000", 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070852199999999995 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.845655+0000", 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015239609999999999 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.846207+0000", 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000551933 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.846301+0000", 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4511000000000004e-05 2024-09-16T14:07:16.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.846364+0000", 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2595999999999995e-05 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:735 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.852526+0000", 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.46995461700000002, 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039025610000000001, 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 735 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852526+0000", 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852526+0000", 2024-09-16T14:07:16.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852528+0000", 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.88e-06 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852539+0000", 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1063000000000001e-05 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852566+0000", 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6905000000000001e-05 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852575+0000", 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4560000000000006e-06 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852659+0000", 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3877999999999999e-05 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852792+0000", 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132986 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853225+0000", 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043336999999999998 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.855199+0000", 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019738580000000002 2024-09-16T14:07:16.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.856308+0000", 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011088059999999999 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.856383+0000", 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5136999999999996e-05 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.856428+0000", 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5222000000000002e-05 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:736 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.852594+0000", 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.46988651100000001, 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046676510000000001, 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.396 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 736 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852594+0000", 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852594+0000", 2024-09-16T14:07:16.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852595+0000", 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3149999999999999e-06 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852601+0000", 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3329999999999996e-06 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852616+0000", 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4931e-05 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.852638+0000", 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1968999999999999e-05 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853892+0000", 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001254191 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854081+0000", 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018897200000000001 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854530+0000", 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044934599999999999 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.855919+0000", 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001388888 2024-09-16T14:07:16.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.857055+0000", 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001135617 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.857145+0000", 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0830999999999998e-05 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.857262+0000", 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116258 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:739 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.873836+0000", 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.448644194, 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003307801, 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.402 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 739 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.873836+0000", 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.873836+0000", 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.873839+0000", 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2859999999999998e-06 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.873850+0000", 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1439e-05 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.873885+0000", 2024-09-16T14:07:16.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4672999999999998e-05 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.873897+0000", 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2526e-05 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.873987+0000", 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0284999999999994e-05 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874153+0000", 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016575699999999999 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874606+0000", 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00045265000000000002 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.876208+0000", 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001601828 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.876984+0000", 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077628599999999995 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877074+0000", 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0169999999999999e-05 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877144+0000", 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9900999999999994e-05 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.407 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:780 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.961788+0000", 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.36069284200000001, 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033778839999999998, 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 780 2024-09-16T14:07:16.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.961788+0000", 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.961788+0000", 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.961790+0000", 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.21e-06 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.961831+0000", 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0889000000000001e-05 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.961866+0000", 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5469999999999998e-05 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.961879+0000", 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3219e-05 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.961998+0000", 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011835100000000001 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.962159+0000", 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016116000000000001 2024-09-16T14:07:16.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.962813+0000", 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000653679 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.964428+0000", 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016158489999999999 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.965057+0000", 2024-09-16T14:07:16.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062898800000000003 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.965118+0000", 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0476999999999997e-05 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.965166+0000", 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7592000000000001e-05 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:782 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.969919+0000", 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.35256171400000003, 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040718380000000004, 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 782 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.414 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.969919+0000", 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.969919+0000", 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.969921+0000", 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.243e-06 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.969933+0000", 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2233e-05 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.969969+0000", 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5952999999999999e-05 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.969985+0000", 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5906e-05 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970110+0000", 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125327 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970268+0000", 2024-09-16T14:07:16.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015786399999999999 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970862+0000", 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059399400000000003 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.973067+0000", 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022047080000000001 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.973850+0000", 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078341100000000002 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.973934+0000", 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3660999999999997e-05 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.973991+0000", 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6538e-05 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:783 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.970010+0000", 2024-09-16T14:07:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.35247082600000001, 2024-09-16T14:07:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046332079999999998, 2024-09-16T14:07:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 783 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970010+0000", 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970010+0000", 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970011+0000", 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.026e-06 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970016+0000", 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970038+0000", 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2549999999999999e-05 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.970045+0000", 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6159999999999998e-06 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971632+0000", 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001586652 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971816+0000", 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018411799999999999 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.972385+0000", 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056961700000000002 2024-09-16T14:07:16.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.974029+0000", 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016441190000000001 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.974518+0000", 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048825099999999998 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.974585+0000", 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7620999999999995e-05 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.974643+0000", 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7568000000000002e-05 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:786 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.978418+0000", 2024-09-16T14:07:16.426 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.34406222800000003, 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038520899999999999, 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 786 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978418+0000", 2024-09-16T14:07:16.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978418+0000", 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978420+0000", 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.871e-06 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978430+0000", 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0325e-05 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978458+0000", 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7912e-05 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978469+0000", 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0254e-05 2024-09-16T14:07:16.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978576+0000", 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107829 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978711+0000", 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000135038 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.979236+0000", 2024-09-16T14:07:16.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052496300000000004 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.981934+0000", 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002697858 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.982013+0000", 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8640999999999995e-05 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.982167+0000", 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015398199999999999 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.982270+0000", 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103417 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:787 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.432 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.978486+0000", 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.34399418399999998, 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045607360000000001, 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 787 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978486+0000", 2024-09-16T14:07:16.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978486+0000", 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978487+0000", 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978492+0000", 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.425e-06 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978513+0000", 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1877000000000001e-05 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.978519+0000", 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8649999999999998e-06 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.980064+0000", 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015445509999999999 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.980200+0000", 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013635200000000001 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.981180+0000", 2024-09-16T14:07:16.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009799889999999999 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.982432+0000", 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00125234 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.982931+0000", 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049821000000000002 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.982994+0000", 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3441000000000005e-05 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983047+0000", 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2902999999999998e-05 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:790 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.986119+0000", 2024-09-16T14:07:16.438 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33636178, 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003268127, 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 790 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986119+0000", 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986119+0000", 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986121+0000", 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0059999999999999e-06 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986131+0000", 2024-09-16T14:07:16.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0360000000000001e-05 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986160+0000", 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9275999999999999e-05 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986176+0000", 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5430999999999998e-05 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986253+0000", 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7119000000000005e-05 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986331+0000", 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7898000000000005e-05 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986660+0000", 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032961999999999998 2024-09-16T14:07:16.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.989243+0000", 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002582394 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.989286+0000", 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3648e-05 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.989332+0000", 2024-09-16T14:07:16.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5769000000000002e-05 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.989387+0000", 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4605999999999999e-05 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:791 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.986227+0000", 2024-09-16T14:07:16.444 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33625332200000002, 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053989279999999999, 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 791 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986227+0000", 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986227+0000", 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986229+0000", 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4339999999999999e-06 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986237+0000", 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1140000000000006e-06 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986256+0000", 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9474e-05 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.986267+0000", 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0934000000000001e-05 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.987273+0000", 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010060259999999999 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.987557+0000", 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000284319 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988483+0000", 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092531600000000003 2024-09-16T14:07:16.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.991000+0000", 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002516739 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.991481+0000", 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048164799999999999 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.991567+0000", 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6019000000000004e-05 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.991626+0000", 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8904999999999999e-05 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.450 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:16.452 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:16.452 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:16.446+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:16.553 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:16.546+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:16.555 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:16.555 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:16.555 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:16.556 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:16.565 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:16.654 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:16.650+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:16.666 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:16.666 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:16.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:16.709 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone1 from 0x8ef8369 (overlap 0x8de46c) to 0x1c60e2 2024-09-16T14:07:16.754 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:16.750+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:16.783 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:16.783 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:16.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:16.783 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:16.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:40 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'19)", 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:54.225896+0000", 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.552542559999999, 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029444900000000001, 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.225896+0000", 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.225896+0000", 2024-09-16T14:07:16.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.225898+0000", 2024-09-16T14:07:16.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6929999999999999e-06 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.225912+0000", 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4032999999999999e-05 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.225942+0000", 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.995e-05 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.225953+0000", 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1323e-05 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.226070+0000", 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117024 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.226110+0000", 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9682999999999999e-05 2024-09-16T14:07:16.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228766+0000", 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026562220000000002 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228841+0000", 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4561999999999995e-05 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:41 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'19)", 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:54.226851+0000", 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.551587974, 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028314059999999999, 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.226851+0000", 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.226851+0000", 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.226852+0000", 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.497e-06 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.226944+0000", 2024-09-16T14:07:16.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1351e-05 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.226978+0000", 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4709000000000003e-05 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.226990+0000", 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2150000000000001e-05 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.227068+0000", 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7844999999999998e-05 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.227102+0000", 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3355999999999997e-05 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229580+0000", 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00247854 2024-09-16T14:07:16.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229682+0000", 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101958 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:42 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'24, mlcod=17'19)", 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:54.227581+0000", 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.550857192999999, 2024-09-16T14:07:16.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043464000000000003, 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.227581+0000", 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.227581+0000", 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.227583+0000", 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4500000000000001e-06 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.227596+0000", 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2857000000000001e-05 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.227622+0000", 2024-09-16T14:07:16.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6642e-05 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.227637+0000", 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.507e-05 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228066+0000", 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00042881100000000003 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228109+0000", 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2842000000000001e-05 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.231829+0000", 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037197430000000002 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.231928+0000", 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8985000000000002e-05 2024-09-16T14:07:16.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:43 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'25, mlcod=17'20)", 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:54.228695+0000", 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.549743166999999, 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003305756, 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228695+0000", 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228695+0000", 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228697+0000", 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228711+0000", 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4153e-05 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228734+0000", 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3502999999999998e-05 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228748+0000", 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.382e-05 2024-09-16T14:07:16.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228916+0000", 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016749899999999999 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.228950+0000", 2024-09-16T14:07:16.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4032999999999998e-05 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.231934+0000", 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029843389999999999 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.232001+0000", 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7129000000000001e-05 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:44 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'26, mlcod=17'20)", 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:54.229620+0000", 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.548818654000002, 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032646440000000001, 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229620+0000", 2024-09-16T14:07:16.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229620+0000", 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229621+0000", 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.294e-06 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229700+0000", 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9121000000000004e-05 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229733+0000", 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2549999999999998e-05 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229749+0000", 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6325999999999999e-05 2024-09-16T14:07:16.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.229978+0000", 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022882800000000001 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.230052+0000", 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4286e-05 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.232735+0000", 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026830320000000001 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.232885+0000", 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149207 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:73 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'51, mlcod=17'47)", 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:54.783289+0000", 2024-09-16T14:07:16.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.995149618999999, 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003033179, 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.783289+0000", 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.783289+0000", 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.783290+0000", 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.783301+0000", 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0195e-05 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.783330+0000", 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.921e-05 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.783339+0000", 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2369999999999998e-06 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.783438+0000", 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9133000000000001e-05 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.783482+0000", 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3588999999999999e-05 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.786235+0000", 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027528299999999999 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.786322+0000", 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7618e-05 2024-09-16T14:07:16.811 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:74 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'52, mlcod=17'48)", 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:54.784267+0000", 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.994171388000002, 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029151770000000001, 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.784267+0000", 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.784267+0000", 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.784268+0000", 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.02e-06 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.784279+0000", 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0903999999999999e-05 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.784297+0000", 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.749e-05 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.784310+0000", 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.329e-05 2024-09-16T14:07:16.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.784401+0000", 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1199999999999994e-05 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.784447+0000", 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5494999999999997e-05 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.787124+0000", 2024-09-16T14:07:16.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026774809999999998 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:54.787182+0000", 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8297000000000001e-05 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:89 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'63, mlcod=17'60)", 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:55.579358+0000", 2024-09-16T14:07:16.816 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.199080636000001, 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002840445, 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.579358+0000", 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.579358+0000", 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.579359+0000", 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8700000000000004e-07 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.579376+0000", 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7129000000000002e-05 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.579405+0000", 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8710000000000001e-05 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.579417+0000", 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2533e-05 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.579535+0000", 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117241 2024-09-16T14:07:16.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.579571+0000", 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6606000000000002e-05 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582043+0000", 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024718359999999998 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582198+0000", 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000155403 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:90 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'64, mlcod=17'60)", 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:55.580706+0000", 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.197733031999999, 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032429329999999999, 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.821 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.580706+0000", 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.580706+0000", 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.580707+0000", 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5999999999999999e-06 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.580830+0000", 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012276100000000001 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.580862+0000", 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1559000000000001e-05 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.580875+0000", 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3825999999999999e-05 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.580975+0000", 2024-09-16T14:07:16.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9514999999999998e-05 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.581013+0000", 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7738999999999999e-05 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583888+0000", 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028751779999999999 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583949+0000", 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0754999999999997e-05 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:91 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'65, mlcod=17'60)", 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:55.581961+0000", 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.196477391999998, 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049794319999999998, 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.826 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.581961+0000", 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.581961+0000", 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.581963+0000", 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6559999999999999e-06 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.581975+0000", 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.256e-05 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582007+0000", 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1819000000000001e-05 2024-09-16T14:07:16.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582018+0000", 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1167e-05 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582262+0000", 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000243988 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582364+0000", 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101471 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.586763+0000", 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043991610000000004 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.586941+0000", 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017761 2024-09-16T14:07:16.830 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:92 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'66, mlcod=18'61)", 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:55.582852+0000", 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.195586910999999, 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041958519999999999, 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.831 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582852+0000", 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582852+0000", 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582853+0000", 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.423e-06 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582863+0000", 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0266e-05 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582881+0000", 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7676000000000002e-05 2024-09-16T14:07:16.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582893+0000", 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1857e-05 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.582987+0000", 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4377999999999997e-05 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583017+0000", 2024-09-16T14:07:16.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9699000000000001e-05 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.586951+0000", 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039335849999999999 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.587048+0000", 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6967999999999997e-05 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:93 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'67, mlcod=18'62)", 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:55.583643+0000", 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.194796157999999, 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035524390000000001, 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583643+0000", 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583643+0000", 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583644+0000", 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.463e-06 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583738+0000", 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4424000000000003e-05 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583755+0000", 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6986e-05 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.583825+0000", 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9648999999999997e-05 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.584002+0000", 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017700999999999999 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.584033+0000", 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1195000000000002e-05 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.587117+0000", 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030842140000000001 2024-09-16T14:07:16.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.587195+0000", 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7497999999999996e-05 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4172.0:96 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'70, mlcod=18'67)", 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:06:55.590418+0000", 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.188020226999999, 2024-09-16T14:07:16.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033302359999999999, 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.590418+0000", 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.590418+0000", 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.590420+0000", 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2279999999999999e-06 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.590486+0000", 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5896999999999996e-05 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.590503+0000", 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7931000000000002e-05 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.590510+0000", 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9040000000000003e-06 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.590625+0000", 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011503200000000001 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.590679+0000", 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3644999999999999e-05 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.593652+0000", 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002972513 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:06:55.593749+0000", 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7085999999999998e-05 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:11.352619+0000", 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.4258200470000002, 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030027249999999999, 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.845 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.352619+0000", 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.352619+0000", 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.352621+0000", 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5160000000000001e-06 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.352634+0000", 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2663000000000001e-05 2024-09-16T14:07:16.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.352669+0000", 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5339999999999997e-05 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.352682+0000", 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2568999999999999e-05 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.352844+0000", 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000162263 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.353129+0000", 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028488400000000001 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.353432+0000", 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030325000000000003 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.355006+0000", 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015736840000000001 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.355466+0000", 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046015200000000002 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.355560+0000", 2024-09-16T14:07:16.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4435e-05 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.355621+0000", 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:740 3.1f 3:fd3196f5:::rbd_header.1136e0d3367d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.877526+0000", 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.90091309500000005, 2024-09-16T14:07:16.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029063750000000001, 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 740 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877526+0000", 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877526+0000", 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877528+0000", 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0260000000000002e-06 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877538+0000", 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0491e-05 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877568+0000", 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0045000000000001e-05 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877584+0000", 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5454000000000001e-05 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877706+0000", 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012291400000000001 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.877890+0000", 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018327299999999999 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.878218+0000", 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032797300000000001 2024-09-16T14:07:16.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.879381+0000", 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011627880000000001 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.880286+0000", 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090581799999999997 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.880373+0000", 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6341000000000006e-05 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.880432+0000", 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9252000000000001e-05 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.857 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:797 3.1f 3:fd3196f5:::rbd_header.1136e0d3367d:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.578869+0000", 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.19956929300000001, 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053892519999999998, 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 797 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.578869+0000", 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.578869+0000", 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.578873+0000", 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2540000000000001e-06 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.578889+0000", 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6501999999999999e-05 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.578933+0000", 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3693999999999999e-05 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.578950+0000", 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7234000000000001e-05 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.579118+0000", 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016796399999999999 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.579135+0000", 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6973000000000002e-05 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.582215+0000", 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030796159999999999 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.582331+0000", 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116797 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.582834+0000", 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050220899999999999 2024-09-16T14:07:16.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.583765+0000", 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00093124500000000003 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.584164+0000", 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039947799999999999 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.584219+0000", 2024-09-16T14:07:16.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5078999999999999e-05 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.584259+0000", 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9206999999999999e-05 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:885 3.18 3:195317a3:::rbd_data.1136e0d3367d.000000000000000b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.711232+0000", 2024-09-16T14:07:16.864 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.067206710000000003, 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032624020000000002, 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 885 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711232+0000", 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711232+0000", 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711235+0000", 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7e-06 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711245+0000", 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0285000000000001e-05 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711278+0000", 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2736e-05 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711291+0000", 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3689999999999999e-05 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711390+0000", 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8492000000000005e-05 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711525+0000", 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013547200000000001 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.714494+0000", 2024-09-16T14:07:16.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002969027 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:886 3.1f 3:fdcab294:::rbd_data.1136e0d3367d.000000000000000c:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.711360+0000", 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.067078586999999995, 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055929980000000001, 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 886 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711360+0000", 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711360+0000", 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711361+0000", 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2500000000000001e-06 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711368+0000", 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3520000000000003e-06 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711384+0000", 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6657e-05 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711394+0000", 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9520000000000006e-06 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711520+0000", 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012615599999999999 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711817+0000", 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029630999999999999 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716953+0000", 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051363210000000001 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:890 3.18 3:195317a3:::rbd_data.1136e0d3367d.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.715147+0000", 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.063291192999999996, 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029731850000000002, 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 890 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715147+0000", 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715147+0000", 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715149+0000", 2024-09-16T14:07:16.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.305e-06 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715155+0000", 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9170000000000003e-06 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715172+0000", 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7193999999999998e-05 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715180+0000", 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1480000000000006e-06 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715275+0000", 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4513999999999997e-05 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715441+0000", 2024-09-16T14:07:16.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016608000000000001 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715793+0000", 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035252500000000002 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716967+0000", 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173798 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718048+0000", 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010809140000000001 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718089+0000", 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0825000000000003e-05 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718121+0000", 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1965000000000003e-05 2024-09-16T14:07:16.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:901 3.1f 3:fdcab294:::rbd_data.1136e0d3367d.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.718664+0000", 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.059774355000000001, 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003391406, 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:16.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 901 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718664+0000", 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718664+0000", 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718666+0000", 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0640000000000001e-06 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718675+0000", 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7560000000000005e-06 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:16.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718695+0000", 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718706+0000", 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1460999999999999e-05 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718762+0000", 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5578000000000002e-05 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718862+0000", 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9685000000000005e-05 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.719106+0000", 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024476000000000001 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.719757+0000", 2024-09-16T14:07:16.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00065008100000000005 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.721986+0000", 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022291419999999999 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722025+0000", 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9314999999999999e-05 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:16.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722056+0000", 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0678e-05 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:16.886 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:16.887 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-16T14:07:16.888 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:16.889 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:16.882+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:16.989 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:16.982+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:16.996 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:16.996 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:16.996 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:16.997 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:17.006 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:17.090 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:17.082+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:17.160 INFO:teuthology.orchestra.run.smithi079.stdout:213 trunc from 0x8ef8369 to 0x6105923 2024-09-16T14:07:17.160 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:17.161 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:17.161 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:17.161 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:17.161 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:17.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:17.182 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:17.203 INFO:teuthology.orchestra.run.smithi079.stdout:216 write 0xd753d3e thru 0xd762a9b (0xed5e bytes) 2024-09-16T14:07:17.207 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:17.202+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:17.209 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:17.210 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:17.230 INFO:teuthology.orchestra.run.smithi079.stdout:221 trunc from 0xd762a9c to 0x30f00a0 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4241.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:01.491948+0000", 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.768725571999999, 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020626590000000001, 2024-09-16T14:07:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.491948+0000", 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.491948+0000", 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.491948+0000", 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4700000000000001e-07 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.491963+0000", 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4552999999999999e-05 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.492010+0000", 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6952e-05 2024-09-16T14:07:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.492026+0000", 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6256000000000002e-05 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.492134+0000", 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108551 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.492187+0000", 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2169000000000001e-05 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.493906+0000", 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001719567 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:01.494010+0000", 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104064 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4241.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:02.501907+0000", 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.758765811, 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017495919999999999, 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.501907+0000", 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.501907+0000", 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.501908+0000", 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9599999999999998e-07 2024-09-16T14:07:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.501917+0000", 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8869999999999998e-06 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.501945+0000", 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7671999999999999e-05 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.501963+0000", 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7790000000000001e-05 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.502113+0000", 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015061999999999999 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.502140+0000", 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6622e-05 2024-09-16T14:07:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.503541+0000", 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001400763 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:02.503657+0000", 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011644199999999999 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:686 3.5 e29/23 3:a7139adb:::rbd_data.1136e0d3367d.0000000000000165:head v 29'2, mlcod=0'0)", 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.747314+0000", 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.5133596540000001, 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014072100000000001, 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.747314+0000", 2024-09-16T14:07:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.747314+0000", 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.747314+0000", 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6399999999999998e-07 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.747391+0000", 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7479999999999997e-05 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.747403+0000", 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2031000000000001e-05 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.747411+0000", 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1969999999999999e-06 2024-09-16T14:07:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.747471+0000", 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0937999999999998e-05 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.747498+0000", 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6103000000000001e-05 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.748655+0000", 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011576049999999999 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.748721+0000", 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5592000000000005e-05 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:690 3.4 3:25659a03:::rbd_data.1136e0d3367d.000000000000016d:head [set-alloc-hint object_size 524288 write_size 524288,write 327680~4461] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.754454+0000", 2024-09-16T14:07:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.5062192400000001, 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018781920000000001, 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 690 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754454+0000", 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754454+0000", 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754456+0000", 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5519999999999999e-06 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754467+0000", 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1674e-05 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754487+0000", 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0307000000000001e-05 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754495+0000", 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0960000000000003e-06 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754571+0000", 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5970000000000006e-05 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754715+0000", 2024-09-16T14:07:17.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144043 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.754823+0000", 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010878 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.755886+0000", 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010625960000000001 2024-09-16T14:07:17.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.756256+0000", 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036962200000000001 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.756299+0000", 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3132e-05 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.756332+0000", 2024-09-16T14:07:17.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3420000000000002e-05 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:695 3.4 3:25659a03:::rbd_data.1136e0d3367d.000000000000016d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.773148+0000", 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4875249859999999, 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023759139999999998, 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 695 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773148+0000", 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773148+0000", 2024-09-16T14:07:17.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773150+0000", 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9580000000000001e-06 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773158+0000", 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9729999999999997e-06 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773184+0000", 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.561e-05 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773195+0000", 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1368e-05 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773285+0000", 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9732000000000001e-05 2024-09-16T14:07:17.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773410+0000", 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125227 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.773535+0000", 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012511099999999999 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774517+0000", 2024-09-16T14:07:17.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000982061 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.775357+0000", 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00084015800000000003 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.775453+0000", 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5793999999999998e-05 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.775524+0000", 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0921999999999996e-05 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:693 3.5 e29/23 3:a7139adb:::rbd_data.1136e0d3367d.0000000000000165:head v 29'3, mlcod=28'1)", 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.774076+0000", 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4865968729999999, 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00111045, 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.292 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774076+0000", 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774076+0000", 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774078+0000", 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.658e-06 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774092+0000", 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4164e-05 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774117+0000", 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5232000000000002e-05 2024-09-16T14:07:17.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774129+0000", 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1596999999999999e-05 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774213+0000", 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4305e-05 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.774256+0000", 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2976999999999999e-05 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.775137+0000", 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00088061599999999997 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.775187+0000", 2024-09-16T14:07:17.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9901000000000002e-05 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:749 3.b e29/23 3:d49083f7:::rbd_data.1136e0d3367d.00000000000001a3:head v 29'3, mlcod=28'1)", 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.891660+0000", 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.3690130009999999, 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001654694, 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891660+0000", 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891660+0000", 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891662+0000", 2024-09-16T14:07:17.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891690+0000", 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7389000000000002e-05 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891724+0000", 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4350000000000001e-05 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891736+0000", 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1729000000000001e-05 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891862+0000", 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012683799999999999 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891919+0000", 2024-09-16T14:07:17.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6192e-05 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.893234+0000", 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013156349999999999 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.893315+0000", 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0585000000000002e-05 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:751 3.b e29/23 3:d49083f7:::rbd_data.1136e0d3367d.00000000000001a3:head v 29'4, mlcod=29'2)", 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.902189+0000", 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.3584842189999999, 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015021839999999999, 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902189+0000", 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902189+0000", 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902191+0000", 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.623e-06 2024-09-16T14:07:17.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902203+0000", 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2543e-05 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902236+0000", 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2953000000000001e-05 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902247+0000", 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0993e-05 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902350+0000", 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010305400000000001 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902406+0000", 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6335999999999997e-05 2024-09-16T14:07:17.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.903616+0000", 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00120919 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.903691+0000", 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5492000000000002e-05 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:781 3.15 e29/23 3:ac311a3e:::rbd_data.1136e0d3367d.000000000000003f:head v 29'7, mlcod=29'5)", 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.966385+0000", 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.294288093, 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023047279999999998, 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.966385+0000", 2024-09-16T14:07:17.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.966385+0000", 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.966387+0000", 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.689e-06 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.966444+0000", 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509000000000001e-05 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.966470+0000", 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5568999999999999e-05 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.966479+0000", 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1220000000000001e-06 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.966595+0000", 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115698 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.966662+0000", 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7484999999999995e-05 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.968596+0000", 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019337709999999999 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.968690+0000", 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3885e-05 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:788 3.5 e29/23 3:a395836b:::rbd_data.1136e0d3367d.000000000000008e:head v 29'4, mlcod=29'2)", 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.983621+0000", 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.277052149, 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001714337, 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983621+0000", 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983621+0000", 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983622+0000", 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.336e-06 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983692+0000", 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9480999999999994e-05 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983714+0000", 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1662999999999999e-05 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983722+0000", 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3100000000000001e-06 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983789+0000", 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7014999999999994e-05 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.983822+0000", 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2670999999999999e-05 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.985269+0000", 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001447353 2024-09-16T14:07:17.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.985335+0000", 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6507999999999994e-05 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:884 3.12 3:4de31fa4:::rbd_data.1136e0d3367d.000000000000000a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.54938156400000004, 2024-09-16T14:07:17.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056685160000000002, 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 884 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:17.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711294+0000", 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7669999999999999e-06 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711309+0000", 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4185999999999999e-05 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711391+0000", 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1997999999999997e-05 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711408+0000", 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7782999999999999e-05 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711606+0000", 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019735900000000001 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711909+0000", 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030280699999999999 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716960+0000", 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050516160000000001 2024-09-16T14:07:17.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(3.18 epoch 30/23 rep_tid 69 entries 30'6 (0'0) error 3:195317a3:::rbd_data.1136e0d3367d.000000000000000b:head by client.4406.0:885 0.000000 -2 [r=-2+0b,r=0+0b] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 28'4)", 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.711861+0000", 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.54881212300000004, 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021183109999999999, 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711861+0000", 2024-09-16T14:07:17.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711861+0000", 2024-09-16T14:07:17.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711862+0000", 2024-09-16T14:07:17.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0299999999999999e-07 2024-09-16T14:07:17.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711878+0000", 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201999999999998e-05 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711912+0000", 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3874000000000001e-05 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711932+0000", 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0254999999999998e-05 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712087+0000", 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000155291 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.713979+0000", 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018920860000000001 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:889 3.5 e30/23 3:a0d58f5c:::rbd_data.1136e0d3367d.000000000000000f:head v 30'6, mlcod=29'3)", 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.712668+0000", 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.54800538099999996, 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020432559999999998, 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712668+0000", 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712668+0000", 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712670+0000", 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1270000000000001e-06 2024-09-16T14:07:17.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712688+0000", 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8482999999999999e-05 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712717+0000", 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8356000000000001e-05 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712748+0000", 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0939000000000003e-05 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712838+0000", 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0579000000000001e-05 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712865+0000", 2024-09-16T14:07:17.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7127999999999998e-05 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.714591+0000", 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017252540000000001 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.714711+0000", 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012039 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:894 3.1 3:834411d3:::rbd_data.1136e0d3367d.0000000000000011:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:17.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:17.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.54479132699999999, 2024-09-16T14:07:17.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067046559999999998, 2024-09-16T14:07:17.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 894 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715885+0000", 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9299999999999999e-06 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715897+0000", 2024-09-16T14:07:17.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2099999999999999e-05 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715943+0000", 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5785000000000003e-05 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715970+0000", 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7529999999999999e-05 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716054+0000", 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4041000000000004e-05 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716135+0000", 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0524000000000004e-05 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716338+0000", 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000203196 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717189+0000", 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085096599999999998 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722487+0000", 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052979530000000002 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722541+0000", 2024-09-16T14:07:17.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4268999999999998e-05 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722587+0000", 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5361999999999997e-05 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:890 3.18 e30/23 3:195317a3:::rbd_data.1136e0d3367d.000000000000000b:head v 30'7, mlcod=28'5)", 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.716497+0000", 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.54417626600000002, 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013886160000000001, 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716497+0000", 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716497+0000", 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716498+0000", 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2500000000000001e-06 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716508+0000", 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3770000000000002e-06 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716530+0000", 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2390000000000001e-05 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716541+0000", 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1477e-05 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716613+0000", 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1760000000000004e-05 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716646+0000", 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2375000000000001e-05 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717847+0000", 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001201798 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717886+0000", 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8189000000000003e-05 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:900 3.12 3:4de31fa4:::rbd_data.1136e0d3367d.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.717422+0000", 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.54325071800000002, 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031451640000000002, 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 900 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717422+0000", 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717422+0000", 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717424+0000", 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5e-06 2024-09-16T14:07:17.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717434+0000", 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0328e-05 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717455+0000", 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1098999999999999e-05 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717465+0000", 2024-09-16T14:07:17.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9680000000000001e-06 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717551+0000", 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6036000000000001e-05 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717720+0000", 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016847699999999999 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718057+0000", 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033744200000000003 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.718858+0000", 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00080072900000000002 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.720400+0000", 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015415050000000001 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.720500+0000", 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100098 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.720568+0000", 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7982000000000001e-05 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:906 3.5 e30/23 3:a395836b:::rbd_data.1136e0d3367d.000000000000008e:head v 30'8, mlcod=29'4)", 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.723495+0000", 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.53717821600000004, 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019506250000000001, 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723495+0000", 2024-09-16T14:07:17.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723495+0000", 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723496+0000", 2024-09-16T14:07:17.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4300000000000002e-07 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723515+0000", 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9361e-05 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723533+0000", 2024-09-16T14:07:17.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7995e-05 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723540+0000", 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2130000000000002e-06 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723591+0000", 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0927000000000002e-05 2024-09-16T14:07:17.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723614+0000", 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3105999999999999e-05 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.725413+0000", 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017986949999999999 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.725446+0000", 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2484999999999998e-05 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:902 3.15 e30/23 3:ac311a3e:::rbd_data.1136e0d3367d.000000000000003f:head v 30'9, mlcod=29'6)", 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.723604+0000", 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.53706873899999996, 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024472539999999998, 2024-09-16T14:07:17.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723604+0000", 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723604+0000", 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723605+0000", 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1800000000000002e-07 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723609+0000", 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6840000000000004e-06 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723619+0000", 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.815e-06 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723625+0000", 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9689999999999999e-06 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723685+0000", 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9587999999999999e-05 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723763+0000", 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8325999999999995e-05 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.726003+0000", 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002239848 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.726052+0000", 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8705999999999998e-05 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:903 3.15 e30/23 3:a91832ea:::rbd_data.1136e0d3367d.0000000000000051:head v 30'11, mlcod=29'6)", 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.724685+0000", 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.53598784200000005, 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001706335, 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.724685+0000", 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.724685+0000", 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.724686+0000", 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8199999999999998e-07 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.724695+0000", 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3979999999999999e-06 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.724714+0000", 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9145000000000001e-05 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.724721+0000", 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6629999999999993e-06 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.724766+0000", 2024-09-16T14:07:17.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4458e-05 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.724782+0000", 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5940999999999999e-05 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.726367+0000", 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001584972 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.726392+0000", 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4876e-05 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:920 3.4 3:23617910:::rbd_data.1136e0d3367d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 327680] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.735710+0000", 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.52496320399999996, 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002434585, 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.365 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 920 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.735710+0000", 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.735710+0000", 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.735712+0000", 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6780000000000001e-06 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.735719+0000", 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4850000000000003e-06 2024-09-16T14:07:17.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.735740+0000", 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.069e-05 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.735747+0000", 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5020000000000003e-06 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.735833+0000", 2024-09-16T14:07:17.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6149999999999993e-05 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.736049+0000", 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021509599999999999 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.736317+0000", 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026862899999999999 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.737356+0000", 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001039255 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.737986+0000", 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062943000000000005 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.738072+0000", 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5661000000000005e-05 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.738145+0000", 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3009000000000005e-05 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.371 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:17.373 INFO:teuthology.orchestra.run.smithi079.stdout:223 write 0xbf4a603 thru 0xbf4b5ab (0xfa9 bytes) 2024-09-16T14:07:17.373 INFO:teuthology.orchestra.run.smithi079.stdout:224 read 0xaacd7d3 thru 0xaad8269 (0xaa97 bytes) 2024-09-16T14:07:17.373 INFO:teuthology.orchestra.run.smithi079.stdout:225 read 0x97f320f thru 0x97f40ab (0xe9d bytes) 2024-09-16T14:07:17.373 INFO:teuthology.orchestra.run.smithi079.stdout:226 trunc from 0xbf4b5ac to 0x2156f15 2024-09-16T14:07:17.373 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:17.366+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:17.374 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:17.385 INFO:teuthology.orchestra.run.smithi079.stdout:227 punch from 0xa0f637 to 0xa1a7ab, (0xb174 bytes) 2024-09-16T14:07:17.385 INFO:teuthology.orchestra.run.smithi079.stdout:228 clone 3 order 18 su 8192 sc 10 2024-09-16T14:07:17.474 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:17.470+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:17.475 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:17.475 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:17.475 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:17.475 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:17.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:17.574 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:17.570+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:17.592 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:17.592 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:17.592 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:17.592 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:17.592 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:17.592 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:07:17.601 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:17.675 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:17.670+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:17.699 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:13 3.8 e26/23 3:14a6dda4:::rbd_object_map.113631a28a63:head v 26'1, mlcod=0'0)", 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:11.368040+0000", 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 6.3272649110000003, 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038663970000000001, 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368040+0000", 2024-09-16T14:07:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368040+0000", 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368041+0000", 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2699999999999999e-07 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368052+0000", 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0879999999999999e-05 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368098+0000", 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5429000000000002e-05 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368113+0000", 2024-09-16T14:07:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5016e-05 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368215+0000", 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102394 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368262+0000", 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7453000000000001e-05 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.371771+0000", 2024-09-16T14:07:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035084270000000002 2024-09-16T14:07:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.371907+0000", 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013607100000000001 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:735 3.6 e29/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 29'108, mlcod=29'106)", 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.853455+0000", 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.841850379, 2024-09-16T14:07:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026461089999999998, 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853455+0000", 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853455+0000", 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853457+0000", 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6130000000000001e-06 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853469+0000", 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.219e-05 2024-09-16T14:07:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853500+0000", 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0929000000000002e-05 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853511+0000", 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1283e-05 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853590+0000", 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9419000000000007e-05 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853647+0000", 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6335999999999997e-05 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.855948+0000", 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002301619 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.856101+0000", 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015271999999999999 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:736 3.6 e29/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 29'109, mlcod=29'106)", 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.854687+0000", 2024-09-16T14:07:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.8406183459999998, 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021490469999999998, 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854687+0000", 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854687+0000", 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854689+0000", 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.697e-06 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854704+0000", 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4806e-05 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854735+0000", 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1275999999999997e-05 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854746+0000", 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0803e-05 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854918+0000", 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017240699999999999 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.854988+0000", 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9601e-05 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.856741+0000", 2024-09-16T14:07:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001753632 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.856836+0000", 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4825000000000001e-05 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:739 3.6 e29/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 29'110, mlcod=29'108)", 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.874831+0000", 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.8204746040000002, 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001902408, 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874831+0000", 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874831+0000", 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874832+0000", 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.37e-06 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874842+0000", 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7429999999999997e-06 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874868+0000", 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5902999999999999e-05 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874877+0000", 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7579999999999998e-06 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.874991+0000", 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114838 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.875057+0000", 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5275000000000001e-05 2024-09-16T14:07:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.876621+0000", 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015642029999999999 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.876733+0000", 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112318 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:749 3.b 3:d49083f7:::rbd_data.1136e0d3367d.00000000000001a3:head [set-alloc-hint object_size 524288 write_size 524288,write 87906~8504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.890836+0000", 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.804469278, 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002955226, 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 749 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890836+0000", 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890836+0000", 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890839+0000", 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8229999999999999e-06 2024-09-16T14:07:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890865+0000", 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6152999999999998e-05 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890898+0000", 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2879000000000001e-05 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890911+0000", 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2773e-05 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890993+0000", 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1773000000000005e-05 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891212+0000", 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000219185 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891469+0000", 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025680599999999999 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.892862+0000", 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013933120000000001 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.893586+0000", 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072398000000000004 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.893680+0000", 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4565e-05 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.893791+0000", 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110977 2024-09-16T14:07:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:751 3.b 3:d49083f7:::rbd_data.1136e0d3367d.00000000000001a3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.901387+0000", 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.7939180700000001, 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028494919999999999, 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 751 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901387+0000", 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901387+0000", 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901389+0000", 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0940000000000002e-06 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901401+0000", 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1613e-05 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901436+0000", 2024-09-16T14:07:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4511999999999998e-05 2024-09-16T14:07:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901467+0000", 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1856999999999997e-05 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901583+0000", 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115383 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901759+0000", 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000176393 2024-09-16T14:07:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902011+0000", 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000252134 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.903267+0000", 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001255548 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.903974+0000", 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070696300000000001 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.904119+0000", 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014561399999999999 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.904237+0000", 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011738100000000001 2024-09-16T14:07:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:782 3.6 e29/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 29'118, mlcod=29'116)", 2024-09-16T14:07:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.971080+0000", 2024-09-16T14:07:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.7242250649999999, 2024-09-16T14:07:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024601340000000001, 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971080+0000", 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971080+0000", 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971082+0000", 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.604e-06 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971095+0000", 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.259e-05 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971126+0000", 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1282000000000003e-05 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971137+0000", 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270000000000001e-05 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971229+0000", 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1509999999999996e-05 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971280+0000", 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1799000000000003e-05 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.973449+0000", 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00216898 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.973540+0000", 2024-09-16T14:07:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1099000000000003e-05 2024-09-16T14:07:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:791 3.6 e29/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 29'124, mlcod=29'121)", 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.988609+0000", 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.706696389, 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026732599999999998, 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988609+0000", 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988609+0000", 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988610+0000", 2024-09-16T14:07:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1100000000000005e-07 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988618+0000", 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5969999999999992e-06 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988636+0000", 2024-09-16T14:07:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7510999999999999e-05 2024-09-16T14:07:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988643+0000", 2024-09-16T14:07:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7880000000000001e-06 2024-09-16T14:07:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988720+0000", 2024-09-16T14:07:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7059999999999997e-05 2024-09-16T14:07:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988756+0000", 2024-09-16T14:07:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5760999999999999e-05 2024-09-16T14:07:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.991240+0000", 2024-09-16T14:07:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002484109 2024-09-16T14:07:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.991282+0000", 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2623000000000002e-05 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:880 3.6 e30/23 3:65cfeb1a:::rbd_data.1136e0d3367d.0000000000000006:head v 30'128, mlcod=30'125)", 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.712596+0000", 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.98270981599999996, 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025605939999999998, 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712596+0000", 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712596+0000", 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712597+0000", 2024-09-16T14:07:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.823e-06 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712615+0000", 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7706e-05 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712664+0000", 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9388999999999998e-05 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712681+0000", 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6569999999999999e-05 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712926+0000", 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024498100000000002 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712982+0000", 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5628999999999999e-05 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715011+0000", 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020292819999999999 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715156+0000", 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014521400000000001 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:907 3.6 e30/23 3:6479ea17:::rbd_data.1136e0d3367d.000000000000008f:head v 30'130, mlcod=30'126)", 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.723354+0000", 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.97195114699999996, 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023499089999999999, 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723354+0000", 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723354+0000", 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723356+0000", 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.818e-06 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723369+0000", 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2466000000000001e-05 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723391+0000", 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2238999999999998e-05 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723402+0000", 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0976e-05 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723471+0000", 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9183000000000005e-05 2024-09-16T14:07:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723496+0000", 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5559e-05 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.725644+0000", 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021471820000000001 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.725704+0000", 2024-09-16T14:07:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0486000000000003e-05 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:917 3.6 e30/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 30'136, mlcod=30'132)", 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.738952+0000", 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.95635328500000005, 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020962849999999998, 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.738952+0000", 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.738952+0000", 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.738953+0000", 2024-09-16T14:07:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.296e-06 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.738962+0000", 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4239999999999993e-06 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.738983+0000", 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1019e-05 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.738992+0000", 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0389999999999993e-06 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.739040+0000", 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8139000000000001e-05 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.739065+0000", 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4766e-05 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740975+0000", 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019101509999999999 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.741048+0000", 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3450999999999999e-05 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:918 3.6 e30/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 30'137, mlcod=30'134)", 2024-09-16T14:07:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.740936+0000", 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.95436926499999997, 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029995500000000001, 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740936+0000", 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740936+0000", 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740937+0000", 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.06e-06 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740953+0000", 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5829e-05 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740969+0000", 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6226e-05 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740981+0000", 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.201e-05 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.741092+0000", 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111093 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.741131+0000", 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8383999999999997e-05 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.743890+0000", 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027588719999999999 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.743936+0000", 2024-09-16T14:07:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1041 3.b 3:d3284eff:::rbd_data.11365ca8c0d.0000000000000031:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.170907+0000", 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.52439800700000005, 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001875268, 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1041 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170907+0000", 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170907+0000", 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170911+0000", 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2739999999999999e-06 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170922+0000", 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0956e-05 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170951+0000", 2024-09-16T14:07:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9201999999999999e-05 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170961+0000", 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0359e-05 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.171040+0000", 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8973000000000004e-05 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.171189+0000", 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000148887 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.171319+0000", 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012998299999999999 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.172191+0000", 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00087150699999999999 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.172633+0000", 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000442936 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.172732+0000", 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8963999999999996e-05 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.172783+0000", 2024-09-16T14:07:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0226999999999999e-05 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1062 3.b 3:d2620549:::rbd_data.11365ca8c0d.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 343358~60766] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.205224+0000", 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.490081035, 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003075225, 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1062 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205224+0000", 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205224+0000", 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205227+0000", 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4779999999999998e-06 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205430+0000", 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020357799999999999 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205468+0000", 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7493000000000002e-05 2024-09-16T14:07:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205491+0000", 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2722e-05 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205579+0000", 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8214999999999995e-05 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205784+0000", 2024-09-16T14:07:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020552499999999999 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.206006+0000", 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022118100000000001 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.207753+0000", 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017474260000000001 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.208145+0000", 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000391956 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.208229+0000", 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4080999999999997e-05 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.208300+0000", 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0569999999999997e-05 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1068 3.b 3:d3284eff:::rbd_data.11365ca8c0d.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.233409+0000", 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.461896678, 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042937849999999996, 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1068 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233409+0000", 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233409+0000", 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233412+0000", 2024-09-16T14:07:17.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2600000000000001e-06 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233429+0000", 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7146000000000002e-05 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233474+0000", 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5346999999999998e-05 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233493+0000", 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8051e-05 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233684+0000", 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019169900000000001 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233921+0000", 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000236598 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234247+0000", 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000326034 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.236457+0000", 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002209866 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.237465+0000", 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010083659999999999 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.237596+0000", 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130903 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.237702+0000", 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010651500000000001 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.782 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1072 3.b 3:d2620549:::rbd_data.11365ca8c0d.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.233527+0000", 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.46177860599999998, 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048508839999999998, 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1072 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.783 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233527+0000", 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233527+0000", 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233529+0000", 2024-09-16T14:07:17.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233536+0000", 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0879999999999997e-06 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233565+0000", 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8906000000000001e-05 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233576+0000", 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1439e-05 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234757+0000", 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011812750000000001 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234949+0000", 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000191929 2024-09-16T14:07:17.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.235253+0000", 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030329400000000001 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.237056+0000", 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018033260000000001 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.238262+0000", 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00120623 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.238327+0000", 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4457999999999998e-05 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.238378+0000", 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1078999999999997e-05 2024-09-16T14:07:17.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1069 3.8 e30/23 3:109ba335:::rbd_data.11365ca8c0d.0000000000000032:head v 30'34, mlcod=30'32)", 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.234116+0000", 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.46118919600000002, 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019036739999999999, 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234116+0000", 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234116+0000", 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234117+0000", 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2410000000000001e-06 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234131+0000", 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3725e-05 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234145+0000", 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3512e-05 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234154+0000", 2024-09-16T14:07:17.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0890000000000006e-06 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234341+0000", 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018702399999999999 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234365+0000", 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3969999999999999e-05 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.235926+0000", 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015614649999999999 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.236020+0000", 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3647999999999995e-05 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1100 3.6 e30/23 3:637d63d5:::rbd_data.11365ca8c0d.000000000000001d:head v 30'140, mlcod=30'138)", 2024-09-16T14:07:17.793 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.253290+0000", 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.44201534199999998, 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021437520000000001, 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.253290+0000", 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.253290+0000", 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.253291+0000", 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2399999999999998e-07 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.253296+0000", 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.694e-06 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.253308+0000", 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1394e-05 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.253313+0000", 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3959999999999997e-06 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.253409+0000", 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6386999999999997e-05 2024-09-16T14:07:17.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.253435+0000", 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5485000000000001e-05 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255373+0000", 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019377159999999999 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255434+0000", 2024-09-16T14:07:17.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1155999999999995e-05 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1165 3.8 e30/23 3:158a6710:::rbd_data.11365ca8c0d.000000000000001c:head v 30'36, mlcod=30'34)", 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.356578+0000", 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33872692700000001, 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034399999999999999, 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356578+0000", 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356578+0000", 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356580+0000", 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8360000000000001e-06 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356600+0000", 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9324e-05 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356634+0000", 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4756999999999999e-05 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356650+0000", 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5159e-05 2024-09-16T14:07:17.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356719+0000", 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9188000000000002e-05 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356744+0000", 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5626000000000002e-05 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.359957+0000", 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032128159999999998 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360018+0000", 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1294e-05 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1167 3.6 e30/23 3:637d63d5:::rbd_data.11365ca8c0d.000000000000001d:head v 30'141, mlcod=30'139)", 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.358955+0000", 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.336350223, 2024-09-16T14:07:17.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035207749999999999, 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358955+0000", 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358955+0000", 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358956+0000", 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1299999999999998e-07 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358964+0000", 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5839999999999997e-06 2024-09-16T14:07:17.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358980+0000", 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6118000000000001e-05 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358988+0000", 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4370000000000007e-06 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.359080+0000", 2024-09-16T14:07:17.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2120000000000006e-05 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.359130+0000", 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0058000000000001e-05 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362401+0000", 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003270991 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:17.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362476+0000", 2024-09-16T14:07:17.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4554000000000005e-05 2024-09-16T14:07:17.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:17.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:17.807 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:17.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:17.808 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone2 from 0x2156f15 (overlap 0x2156f15) to 0xa007b9 2024-09-16T14:07:17.809 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:17.802+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:17.810 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:17.841 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-16T14:07:17.863 INFO:teuthology.orchestra.run.smithi079.stdout:231 read 0x1efe88f thru 0x1f04f90 (0x6702 bytes) 2024-09-16T14:07:17.906 INFO:teuthology.orchestra.run.smithi079.stdout:234 writesame 0xe20340a thru 0xe20c56c data_size 0xd(0x9163 bytes) 2024-09-16T14:07:17.906 INFO:teuthology.orchestra.run.smithi079.stdout:235 read 0xc0c6d59 thru 0xc0c9cda (0x2f82 bytes) 2024-09-16T14:07:17.906 INFO:teuthology.orchestra.run.smithi079.stdout:238 write 0x7feb591 thru 0x7ff2e33 (0x78a3 bytes) 2024-09-16T14:07:17.909 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:17.902+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:17.918 INFO:teuthology.orchestra.run.smithi079.stdout:239 punch from 0xd42e0c5 to 0xd431092, (0x2fcd bytes) 2024-09-16T14:07:17.918 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:17.941 INFO:teuthology.orchestra.run.smithi079.stdout:246 trunc from 0xe20c56d to 0x132c18d 2024-09-16T14:07:17.941 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:17.942 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:17.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:17.960 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:17.991 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:18.001 INFO:teuthology.orchestra.run.smithi079.stdout:247 writesame 0xb2f09bb thru 0xb2fb2d7 data_size 0x1(0xa91d bytes) 2024-09-16T14:07:18.001 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:18.001 INFO:teuthology.orchestra.run.smithi079.stdout:250 write 0x7f62307 thru 0x7f6d4f4 (0xb1ee bytes) 2024-09-16T14:07:18.009 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:18.002+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:18.017 INFO:teuthology.orchestra.run.smithi079.stdout:252 read 0x7bc315 thru 0x7c8bf8 (0xc8e4 bytes) 2024-09-16T14:07:18.018 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:18.024 INFO:teuthology.orchestra.run.smithi079.stdout:253 trunc from 0xb2fb2d8 to 0x8e77326 2024-09-16T14:07:18.044 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:18.048 INFO:teuthology.orchestra.run.smithi079.stdout:254 write 0x1bbd6f7 thru 0x1bccdd3 (0xf6dd bytes) 2024-09-16T14:07:18.065 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:18.066 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:18.067 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:18.074 INFO:teuthology.orchestra.run.smithi079.stdout:255 trunc from 0x8e77326 to 0x5f7bbeb 2024-09-16T14:07:18.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:18.087 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:18.107 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:18.111 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:18.106+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:18.112 INFO:teuthology.orchestra.run.smithi079.stdout:258 write 0x9741c5b thru 0x9744027 (0x23cd bytes) 2024-09-16T14:07:18.129 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:18.130 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:18.130 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:18.131 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:18.131 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:18.131 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:18.131 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:18.131 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:18.131 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:18.137 INFO:teuthology.orchestra.run.smithi079.stdout:259 write 0xe1ae58e thru 0xe1b6684 (0x80f7 bytes) 2024-09-16T14:07:18.154 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:18.154 INFO:teuthology.orchestra.run.smithi079.stdout:263 read 0x83dc455 thru 0x83eba22 (0xf5ce bytes) 2024-09-16T14:07:18.154 INFO:teuthology.orchestra.run.smithi079.stdout:264 write 0x3ed9929 thru 0x3ee39e9 (0xa0c1 bytes) 2024-09-16T14:07:18.171 INFO:teuthology.orchestra.run.smithi079.stdout:265 write 0x167c597 thru 0x168442e (0x7e98 bytes) 2024-09-16T14:07:18.185 INFO:teuthology.orchestra.run.smithi079.stdout:266 write 0xfaff27 thru 0xfb155d (0x1637 bytes) 2024-09-16T14:07:18.191 INFO:teuthology.orchestra.run.smithi079.stdout:267 write 0xbff3d6 thru 0xc085f9 (0x9224 bytes) 2024-09-16T14:07:18.196 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:18.196 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:18.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:18.208 INFO:teuthology.orchestra.run.smithi079.stdout:268 trunc from 0xe1b6685 to 0xb236fa4 2024-09-16T14:07:18.226 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:18.226 INFO:teuthology.orchestra.run.smithi079.stdout:270 read 0x59e5a12 thru 0x59e8c3e (0x322d bytes) 2024-09-16T14:07:18.226 INFO:teuthology.orchestra.run.smithi079.stdout:271 punch from 0x784ecc4 to 0x785b079, (0xc3b5 bytes) 2024-09-16T14:07:18.226 INFO:teuthology.orchestra.run.smithi079.stdout:272 write 0x3fbf73f thru 0x3fc0751 (0x1013 bytes) 2024-09-16T14:07:18.227 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:18.218+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:18.232 INFO:teuthology.orchestra.run.smithi079.stdout:273 write 0x335216b thru 0x335b0e2 (0x8f78 bytes) 2024-09-16T14:07:18.261 INFO:teuthology.orchestra.run.smithi079.stdout:274 write 0xcfd58cf thru 0xcfdb2b2 (0x59e4 bytes) 2024-09-16T14:07:18.272 INFO:teuthology.orchestra.run.smithi079.stdout:275 writesame 0x8ae4b65 thru 0x8aebb23 data_size 0x1(0x6fbf bytes) 2024-09-16T14:07:18.290 INFO:teuthology.orchestra.run.smithi079.stdout:278 punch from 0xedc046 to 0xee3feb, (0x7fa5 bytes) 2024-09-16T14:07:18.307 INFO:teuthology.orchestra.run.smithi079.stdout:279 trunc from 0xcfdb2b3 to 0x2aca200 2024-09-16T14:07:18.313 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:18.306+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:18.316 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:18.316 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:18.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:18.362 INFO:teuthology.orchestra.run.smithi079.stdout:281 writesame 0xeb3444d thru 0xeb35185 data_size 0x1(0xd39 bytes) 2024-09-16T14:07:18.362 INFO:teuthology.orchestra.run.smithi079.stdout:282 write 0x35b7978 thru 0x35bd01b (0x56a4 bytes) 2024-09-16T14:07:18.377 INFO:teuthology.orchestra.run.smithi079.stdout:283 write 0x740502f thru 0x7407c0b (0x2bdd bytes) 2024-09-16T14:07:18.400 INFO:teuthology.orchestra.run.smithi079.stdout:287 trunc from 0xeb35186 to 0x5f3ad63 2024-09-16T14:07:18.414 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:18.410+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:18.418 INFO:teuthology.orchestra.run.smithi079.stdout:289 write 0x1225ed2 thru 0x122aa39 (0x4b68 bytes) 2024-09-16T14:07:18.422 INFO:teuthology.orchestra.run.smithi079.stdout:290 punch from 0x4d9154c to 0x4d9d83d, (0xc2f1 bytes) 2024-09-16T14:07:18.422 INFO:teuthology.orchestra.run.smithi079.stdout:291 read 0x9fdd9 thru 0xa0369 (0x591 bytes) 2024-09-16T14:07:18.433 INFO:teuthology.orchestra.run.smithi079.stdout:292 trunc from 0x5f3ad63 to 0xbdafda9 2024-09-16T14:07:18.443 INFO:teuthology.orchestra.run.smithi079.stdout:295 writesame 0x90f6cff thru 0x9106b87 data_size 0x1(0xfe89 bytes) 2024-09-16T14:07:18.472 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:18.472 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:18.477 INFO:teuthology.orchestra.run.smithi079.stdout:296 read 0x7c4442 thru 0x7d2660 (0xe21f bytes) 2024-09-16T14:07:18.480 INFO:teuthology.orchestra.run.smithi079.stdout:297 read 0x62e7503 thru 0x62ececf (0x59cd bytes) 2024-09-16T14:07:18.480 INFO:teuthology.orchestra.run.smithi079.stdout:298 read 0x974d5cb thru 0x9754a33 (0x7469 bytes) 2024-09-16T14:07:18.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:18.495 INFO:teuthology.orchestra.run.smithi079.stdout:299 trunc from 0xbdafda9 to 0x1f4b853 2024-09-16T14:07:18.546 INFO:teuthology.orchestra.run.smithi079.stdout:301 write 0xc6b6570 thru 0xc6c4a78 (0xe509 bytes) 2024-09-16T14:07:18.546 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:18.542+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:18.570 INFO:teuthology.orchestra.run.smithi079.stdout:302 writesame 0x4ce66d5 thru 0x4cf20b9 data_size 0x3(0xb9e5 bytes) 2024-09-16T14:07:18.590 INFO:teuthology.orchestra.run.smithi079.stdout:304 read 0x79bdd38 thru 0x79cca14 (0xecdd bytes) 2024-09-16T14:07:18.590 INFO:teuthology.orchestra.run.smithi079.stdout:305 punch from 0x7b0919 to 0x7b74b2, (0x6b99 bytes) 2024-09-16T14:07:18.590 INFO:teuthology.orchestra.run.smithi079.stdout:306 read 0x7c03a69 thru 0x7c10ea4 (0xd43c bytes) 2024-09-16T14:07:18.597 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:07:18.604 INFO:teuthology.orchestra.run.smithi079.stdout:308 write 0xcd786b9 thru 0xcd8120a (0x8b52 bytes) 2024-09-16T14:07:18.608 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:18.608 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:18.615 INFO:teuthology.orchestra.run.smithi079.stdout:309 read 0x5392878 thru 0x5393472 (0xbfb bytes) 2024-09-16T14:07:18.615 INFO:teuthology.orchestra.run.smithi079.stdout:310 read 0x81ee4cf thru 0x81f385a (0x538c bytes) 2024-09-16T14:07:18.616 INFO:teuthology.orchestra.run.smithi079.stdout:312 read 0x4b2b09e thru 0x4b2c21b (0x117e bytes) 2024-09-16T14:07:18.616 INFO:teuthology.orchestra.run.smithi079.stdout:313 writesame 0x36b4d99 thru 0x36bfbf8 data_size 0xa0(0xae60 bytes) 2024-09-16T14:07:18.616 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:18.610+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:18.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:18.629 INFO:teuthology.orchestra.run.smithi079.stdout:314 write 0xbb79e91 thru 0xbb7fac7 (0x5c37 bytes) 2024-09-16T14:07:18.640 INFO:teuthology.orchestra.run.smithi079.stdout:315 write 0xc553973 thru 0xc55a767 (0x6df5 bytes) 2024-09-16T14:07:18.652 INFO:teuthology.orchestra.run.smithi079.stdout:316 writesame 0x61d47f2 thru 0x61d53d6 data_size 0x15(0xbe5 bytes) 2024-09-16T14:07:18.658 INFO:teuthology.orchestra.run.smithi079.stdout:320 read 0x88871a7 thru 0x889007f (0x8ed9 bytes) 2024-09-16T14:07:18.658 INFO:teuthology.orchestra.run.smithi079.stdout:321 read 0x7610a24 thru 0x761185a (0xe37 bytes) 2024-09-16T14:07:18.658 INFO:teuthology.orchestra.run.smithi079.stdout:326 read 0xb17ff71 thru 0xb18afa6 (0xb036 bytes) 2024-09-16T14:07:18.658 INFO:teuthology.orchestra.run.smithi079.stdout:327 punch from 0x23b1aa4 to 0x23bc204, (0xa760 bytes) 2024-09-16T14:07:18.658 INFO:teuthology.orchestra.run.smithi079.stdout:329 read 0x37dad1d thru 0x37e08f6 (0x5bda bytes) 2024-09-16T14:07:18.658 INFO:teuthology.orchestra.run.smithi079.stdout:330 write 0xa482e50 thru 0xa48e817 (0xb9c8 bytes) 2024-09-16T14:07:18.672 INFO:teuthology.orchestra.run.smithi079.stdout:331 write 0x54e1be6 thru 0x54e73e9 (0x5804 bytes) 2024-09-16T14:07:18.702 INFO:teuthology.orchestra.run.smithi079.stdout:332 trunc from 0xcd8120b to 0x21a2355 2024-09-16T14:07:18.716 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:18.710+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:18.727 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:18.728 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:18.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:18.746 INFO:teuthology.orchestra.run.smithi079.stdout:333 read 0xcdec70 thru 0xce568f (0x6a20 bytes) 2024-09-16T14:07:18.747 INFO:teuthology.orchestra.run.smithi079.stdout:335 punch from 0x2064589 to 0x206882b, (0x42a2 bytes) 2024-09-16T14:07:18.748 INFO:teuthology.orchestra.run.smithi079.stdout:336 read 0x1731f21 thru 0x1732ace (0xbae bytes) 2024-09-16T14:07:18.748 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:18.748 INFO:teuthology.orchestra.run.smithi079.stdout:341 read 0x74cc4b thru 0x755635 (0x89eb bytes) 2024-09-16T14:07:18.775 INFO:teuthology.orchestra.run.smithi079.stdout:342 writesame 0x667260a thru 0x6678ff9 data_size 0x71(0x69f0 bytes) 2024-09-16T14:07:18.775 INFO:teuthology.orchestra.run.smithi079.stdout:343 write 0x538a062 thru 0x53954d0 (0xb46f bytes) 2024-09-16T14:07:18.791 INFO:teuthology.orchestra.run.smithi079.stdout:345 read 0x650cecc thru 0x6519682 (0xc7b7 bytes) 2024-09-16T14:07:18.791 INFO:teuthology.orchestra.run.smithi079.stdout:346 read 0x3d452b3 thru 0x3d53261 (0xdfaf bytes) 2024-09-16T14:07:18.791 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:18.791 INFO:teuthology.orchestra.run.smithi079.stdout:348 read 0x2c210e7 thru 0x2c2dd68 (0xcc82 bytes) 2024-09-16T14:07:18.812 INFO:teuthology.orchestra.run.smithi079.stdout:349 write 0xccbd75e thru 0xccbed82 (0x1625 bytes) 2024-09-16T14:07:18.817 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:18.810+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:18.821 INFO:teuthology.orchestra.run.smithi079.stdout:350 punch from 0x366697 to 0x374dfd, (0xe766 bytes) 2024-09-16T14:07:18.821 INFO:teuthology.orchestra.run.smithi079.stdout:352 write 0x7297fed thru 0x729d0cd (0x50e1 bytes) 2024-09-16T14:07:18.838 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:18.838 INFO:teuthology.orchestra.run.smithi079.stdout:358 write 0x564bb9 thru 0x56cb64 (0x7fac bytes) 2024-09-16T14:07:18.847 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:18.847 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:18.854 INFO:teuthology.orchestra.run.smithi079.stdout:359 write 0x826798 thru 0x83561f (0xee88 bytes) 2024-09-16T14:07:18.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:18.862 INFO:teuthology.orchestra.run.smithi079.stdout:361 read 0xc8bf0da thru 0xc8cba88 (0xc9af bytes) 2024-09-16T14:07:18.862 INFO:teuthology.orchestra.run.smithi079.stdout:363 write 0xb5aebe5 thru 0xb5afd77 (0x1193 bytes) 2024-09-16T14:07:18.864 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:18.875 INFO:teuthology.orchestra.run.smithi079.stdout:365 write 0xe48f9b8 thru 0xe49f550 (0xfb99 bytes) 2024-09-16T14:07:18.897 INFO:teuthology.orchestra.run.smithi079.stdout:366 writesame 0x86873be thru 0x8695427 data_size 0x4b(0xe06a bytes) 2024-09-16T14:07:18.918 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:18.910+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:18.930 INFO:teuthology.orchestra.run.smithi079.stdout:367 trunc from 0xe49f551 to 0x5168d45 2024-09-16T14:07:18.955 INFO:teuthology.orchestra.run.smithi079.stdout:368 punch from 0x130a7d7 to 0x1312540, (0x7d69 bytes) 2024-09-16T14:07:18.955 INFO:teuthology.orchestra.run.smithi079.stdout:370 read 0x69767a thru 0x69ae04 (0x378b bytes) 2024-09-16T14:07:18.955 INFO:teuthology.orchestra.run.smithi079.stdout:371 punch from 0x165b910 to 0x166685c, (0xaf4c bytes) 2024-09-16T14:07:18.963 INFO:teuthology.orchestra.run.smithi079.stdout:372 writesame 0x51c9a51 thru 0x51d35d4 data_size 0x4(0x9b84 bytes) 2024-09-16T14:07:18.963 INFO:teuthology.orchestra.run.smithi079.stdout:374 punch from 0x1c80c77 to 0x1c882e3, (0x766c bytes) 2024-09-16T14:07:18.963 INFO:teuthology.orchestra.run.smithi079.stdout:375 read 0x3c09cfd thru 0x3c19903 (0xfc07 bytes) 2024-09-16T14:07:18.963 INFO:teuthology.orchestra.run.smithi079.stdout:376 read 0x147c67d thru 0x1482a54 (0x63d8 bytes) 2024-09-16T14:07:18.963 INFO:teuthology.orchestra.run.smithi079.stdout:377 read 0xdf3a71 thru 0xdfa65a (0x6bea bytes) 2024-09-16T14:07:18.964 INFO:teuthology.orchestra.run.smithi079.stdout:379 write 0x447f7bc thru 0x44885a2 (0x8de7 bytes) 2024-09-16T14:07:18.970 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:18.971 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:18.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:18.982 INFO:teuthology.orchestra.run.smithi079.stdout:382 punch from 0x4b8508a to 0x4b87776, (0x26ec bytes) 2024-09-16T14:07:18.982 INFO:teuthology.orchestra.run.smithi079.stdout:384 read 0x48c3f0f thru 0x48c9d38 (0x5e2a bytes) 2024-09-16T14:07:18.998 INFO:teuthology.orchestra.run.smithi079.stdout:385 write 0x99e6afa thru 0x99f077c (0x9c83 bytes) 2024-09-16T14:07:19.016 INFO:teuthology.orchestra.run.smithi079.stdout:386 write 0x1b68440 thru 0x1b77a67 (0xf628 bytes) 2024-09-16T14:07:19.019 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:19.010+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:19.036 INFO:teuthology.orchestra.run.smithi079.stdout:390 trunc from 0x99f077d to 0x7bc6c0c 2024-09-16T14:07:19.053 INFO:teuthology.orchestra.run.smithi079.stdout:391 punch from 0x3b1782a to 0x3b2689c, (0xf072 bytes) 2024-09-16T14:07:19.053 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.073 INFO:teuthology.orchestra.run.smithi079.stdout:393 writesame 0xceb175a thru 0xcebb836 data_size 0x35(0xa0dd bytes) 2024-09-16T14:07:19.073 INFO:teuthology.orchestra.run.smithi079.stdout:394 read 0x1d43013 thru 0x1d457ad (0x279b bytes) 2024-09-16T14:07:19.073 INFO:teuthology.orchestra.run.smithi079.stdout:395 read 0x3e4d37 thru 0x3f11f8 (0xc4c2 bytes) 2024-09-16T14:07:19.074 INFO:teuthology.orchestra.run.smithi079.stdout:396 read 0x5aac2ba thru 0x5ab34eb (0x7232 bytes) 2024-09-16T14:07:19.076 INFO:teuthology.orchestra.run.smithi079.stdout:398 punch from 0xa0fb424 to 0xa1005e9, (0x51c5 bytes) 2024-09-16T14:07:19.076 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.076 INFO:teuthology.orchestra.run.smithi079.stdout:401 read 0xc5b20cb thru 0xc5b26cd (0x603 bytes) 2024-09-16T14:07:19.077 INFO:teuthology.orchestra.run.smithi079.stdout:403 write 0x2ec5830 thru 0x2ecfa1f (0xa1f0 bytes) 2024-09-16T14:07:19.088 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:19.089 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:19.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:19.100 INFO:teuthology.orchestra.run.smithi079.stdout:407 punch from 0x657f7d1 to 0x6587fb2, (0x87e1 bytes) 2024-09-16T14:07:19.116 INFO:teuthology.orchestra.run.smithi079.stdout:408 writesame 0xe54e574 thru 0xe5582c8 data_size 0x1(0x9d55 bytes) 2024-09-16T14:07:19.117 INFO:teuthology.orchestra.run.smithi079.stdout:409 punch from 0x88a75a2 to 0x88b08ed, (0x934b bytes) 2024-09-16T14:07:19.117 INFO:teuthology.orchestra.run.smithi079.stdout:410 writesame 0x2f68063 thru 0x2f6cf7d data_size 0xb(0x4f1b bytes) 2024-09-16T14:07:19.120 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:19.114+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:19.120 INFO:teuthology.orchestra.run.smithi079.stdout:411 read 0x81863d7 thru 0x818f0db (0x8d05 bytes) 2024-09-16T14:07:19.120 INFO:teuthology.orchestra.run.smithi079.stdout:412 punch from 0xd4db2c to 0xd52ec5, (0x5399 bytes) 2024-09-16T14:07:19.121 INFO:teuthology.orchestra.run.smithi079.stdout:413 writesame 0x8c5f3b6 thru 0x8c689d0 data_size 0x1(0x961b bytes) 2024-09-16T14:07:19.157 INFO:teuthology.orchestra.run.smithi079.stdout:414 trunc from 0xe5582c9 to 0x5f05008 2024-09-16T14:07:19.183 INFO:teuthology.orchestra.run.smithi079.stdout:415 writesame 0x7449b1 thru 0x74b3b8 data_size 0x2(0x6a08 bytes) 2024-09-16T14:07:19.196 INFO:teuthology.orchestra.run.smithi079.stdout:416 read 0x7d5fd4 thru 0x7dbea4 (0x5ed1 bytes) 2024-09-16T14:07:19.197 INFO:teuthology.orchestra.run.smithi079.stdout:417 writesame 0x598afdc thru 0x5994c82 data_size 0x11(0x9ca7 bytes) 2024-09-16T14:07:19.216 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.217 INFO:teuthology.orchestra.run.smithi079.stdout:419 read 0x21f909a thru 0x22014e7 (0x844e bytes) 2024-09-16T14:07:19.217 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.221 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:19.214+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:19.221 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:19.222 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:19.228 INFO:teuthology.orchestra.run.smithi079.stdout:421 writesame 0x775f3e1 thru 0x776365f data_size 0x1(0x427f bytes) 2024-09-16T14:07:19.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:19.250 INFO:teuthology.orchestra.run.smithi079.stdout:422 write 0xe8c8c51 thru 0xe8d8211 (0xf5c1 bytes) 2024-09-16T14:07:19.271 INFO:teuthology.orchestra.run.smithi079.stdout:424 punch from 0x3b228c6 to 0x3b2d1a9, (0xa8e3 bytes) 2024-09-16T14:07:19.272 INFO:teuthology.orchestra.run.smithi079.stdout:425 writesame 0xc20ed8a thru 0xc219b51 data_size 0x86(0xadc8 bytes) 2024-09-16T14:07:19.283 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.283 INFO:teuthology.orchestra.run.smithi079.stdout:427 read 0x3cef039 thru 0x3cf2eae (0x3e76 bytes) 2024-09-16T14:07:19.283 INFO:teuthology.orchestra.run.smithi079.stdout:428 write 0xde6d805 thru 0xde75ef6 (0x86f2 bytes) 2024-09-16T14:07:19.294 INFO:teuthology.orchestra.run.smithi079.stdout:429 writesame 0xa8f174b thru 0xa8f9c4f data_size 0x1(0x8505 bytes) 2024-09-16T14:07:19.310 INFO:teuthology.orchestra.run.smithi079.stdout:431 read 0x9a6d0ac thru 0x9a7bcee (0xec43 bytes) 2024-09-16T14:07:19.310 INFO:teuthology.orchestra.run.smithi079.stdout:432 read 0x7b5a6f8 thru 0x7b5a8f9 (0x202 bytes) 2024-09-16T14:07:19.310 INFO:teuthology.orchestra.run.smithi079.stdout:434 writesame 0x423e8df thru 0x4246152 data_size 0x1a(0x7874 bytes) 2024-09-16T14:07:19.322 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:19.318+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:19.323 INFO:teuthology.orchestra.run.smithi079.stdout:438 read 0x7de4ea4 thru 0x7de8987 (0x3ae4 bytes) 2024-09-16T14:07:19.323 INFO:teuthology.orchestra.run.smithi079.stdout:439 read 0xd923269 thru 0xd92d4fa (0xa292 bytes) 2024-09-16T14:07:19.323 INFO:teuthology.orchestra.run.smithi079.stdout:440 write 0xb722435 thru 0xb7269e8 (0x45b4 bytes) 2024-09-16T14:07:19.332 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.332 INFO:teuthology.orchestra.run.smithi079.stdout:443 read 0x79c43a4 thru 0x79c5e3c (0x1a99 bytes) 2024-09-16T14:07:19.332 INFO:teuthology.orchestra.run.smithi079.stdout:444 writesame 0x668b332 thru 0x66939f3 data_size 0x2f(0x86c2 bytes) 2024-09-16T14:07:19.345 INFO:teuthology.orchestra.run.smithi079.stdout:446 write 0x1f938ad thru 0x1f951fd (0x1951 bytes) 2024-09-16T14:07:19.352 INFO:teuthology.orchestra.run.smithi079.stdout:451 read 0x80029e0 thru 0x800af94 (0x85b5 bytes) 2024-09-16T14:07:19.352 INFO:teuthology.orchestra.run.smithi079.stdout:453 read 0x53df4cb thru 0x53ed0cc (0xdc02 bytes) 2024-09-16T14:07:19.352 INFO:teuthology.orchestra.run.smithi079.stdout:456 writesame 0x3d846b3 thru 0x3d8de98 data_size 0x3(0x97e6 bytes) 2024-09-16T14:07:19.359 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:19.359 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:19.372 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:19.380 INFO:teuthology.orchestra.run.smithi079.stdout:458 trunc from 0xe8d8212 to 0x53c9e5a 2024-09-16T14:07:19.414 INFO:teuthology.orchestra.run.smithi079.stdout:460 trunc from 0x53c9e5a to 0x7bde807 2024-09-16T14:07:19.423 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:19.414+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:19.423 INFO:teuthology.orchestra.run.smithi079.stdout:461 read 0x379cc1c thru 0x37a6c82 (0xa067 bytes) 2024-09-16T14:07:19.423 INFO:teuthology.orchestra.run.smithi079.stdout:462 write 0x72195b8 thru 0x7223ee1 (0xa92a bytes) 2024-09-16T14:07:19.450 INFO:teuthology.orchestra.run.smithi079.stdout:463 write 0xa62e9c9 thru 0xa63565a (0x6c92 bytes) 2024-09-16T14:07:19.480 INFO:teuthology.orchestra.run.smithi079.stdout:464 writesame 0xe974a8e thru 0xe97ff89 data_size 0x75(0xb4fc bytes) 2024-09-16T14:07:19.480 INFO:teuthology.orchestra.run.smithi079.stdout:465 punch from 0x15add7f to 0x15b774b, (0x99cc bytes) 2024-09-16T14:07:19.490 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:19.490 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:19.491 INFO:teuthology.orchestra.run.smithi079.stdout:467 trunc from 0xe97ff8a to 0xa5ab6b5 2024-09-16T14:07:19.500 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:19.517 INFO:teuthology.orchestra.run.smithi079.stdout:468 trunc from 0xa5ab6b5 to 0x5250f1e 2024-09-16T14:07:19.524 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:19.518+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:19.540 INFO:teuthology.orchestra.run.smithi079.stdout:469 read 0xe3b88b thru 0xe40eab (0x5621 bytes) 2024-09-16T14:07:19.540 INFO:teuthology.orchestra.run.smithi079.stdout:470 read 0x439db4 thru 0x4492fe (0xf54b bytes) 2024-09-16T14:07:19.563 INFO:teuthology.orchestra.run.smithi079.stdout:471 writesame 0x909963c thru 0x909e53c data_size 0x5(0x4f01 bytes) 2024-09-16T14:07:19.563 INFO:teuthology.orchestra.run.smithi079.stdout:472 write 0x74620f3 thru 0x746808b (0x5f99 bytes) 2024-09-16T14:07:19.576 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.576 INFO:teuthology.orchestra.run.smithi079.stdout:475 read 0x304359f thru 0x3050366 (0xcdc8 bytes) 2024-09-16T14:07:19.576 INFO:teuthology.orchestra.run.smithi079.stdout:476 read 0x5bffa5c thru 0x5c0a690 (0xac35 bytes) 2024-09-16T14:07:19.582 INFO:teuthology.orchestra.run.smithi079.stdout:477 trunc from 0x909e53d to 0xd12f499 2024-09-16T14:07:19.605 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:19.605 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:19.610 INFO:teuthology.orchestra.run.smithi079.stdout:478 trunc from 0xd12f499 to 0x4a11950 2024-09-16T14:07:19.614 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:19.624 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:19.618+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:19.630 INFO:teuthology.orchestra.run.smithi079.stdout:479 read 0x9d51a4 thru 0x9d7631 (0x248e bytes) 2024-09-16T14:07:19.648 INFO:teuthology.orchestra.run.smithi079.stdout:480 write 0x7671d5b thru 0x7674b29 (0x2dcf bytes) 2024-09-16T14:07:19.660 INFO:teuthology.orchestra.run.smithi079.stdout:482 write 0x3c14f6c thru 0x3c1c948 (0x79dd bytes) 2024-09-16T14:07:19.671 INFO:teuthology.orchestra.run.smithi079.stdout:truncating to largest ever: 0xdf60470 2024-09-16T14:07:19.685 INFO:teuthology.orchestra.run.smithi079.stdout:483 trunc from 0x7674b2a to 0xdf60470 2024-09-16T14:07:19.695 INFO:teuthology.orchestra.run.smithi079.stdout:484 punch from 0xb2bca78 to 0xb2c1341, (0x48c9 bytes) 2024-09-16T14:07:19.707 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:19.707 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:19.716 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:19.717 INFO:teuthology.orchestra.run.smithi079.stdout:485 trunc from 0xdf60470 to 0x28f0139 2024-09-16T14:07:19.724 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:19.718+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:19.741 INFO:teuthology.orchestra.run.smithi079.stdout:486 writesame 0x978db5 thru 0x97984b data_size 0x1(0xa97 bytes) 2024-09-16T14:07:19.765 INFO:teuthology.orchestra.run.smithi079.stdout:487 write 0x8e715b3 thru 0x8e74ed2 (0x3920 bytes) 2024-09-16T14:07:19.774 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.778 INFO:teuthology.orchestra.run.smithi079.stdout:490 trunc from 0x8e74ed3 to 0xb26c9cc 2024-09-16T14:07:19.788 INFO:teuthology.orchestra.run.smithi079.stdout:491 write 0x56033f6 thru 0x560c259 (0x8e64 bytes) 2024-09-16T14:07:19.806 INFO:teuthology.orchestra.run.smithi079.stdout:494 writesame 0x2f26261 thru 0x2f334b1 data_size 0x83(0xd251 bytes) 2024-09-16T14:07:19.813 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:19.813 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:19.819 INFO:teuthology.orchestra.run.smithi079.stdout:496 writesame 0x638aabb thru 0x639a136 data_size 0x19(0xf67c bytes) 2024-09-16T14:07:19.824 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:19.825 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:19.818+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:19.844 INFO:teuthology.orchestra.run.smithi079.stdout:498 trunc from 0xb26c9cc to 0x7a23807 2024-09-16T14:07:19.861 INFO:teuthology.orchestra.run.smithi079.stdout:500 writesame 0x23abcb0 thru 0x23b0a87 data_size 0x5e(0x4dd8 bytes) 2024-09-16T14:07:19.875 INFO:teuthology.orchestra.run.smithi079.stdout:501 punch from 0xfb1147 to 0xfb6b04, (0x59bd bytes) 2024-09-16T14:07:19.882 INFO:teuthology.orchestra.run.smithi079.stdout:502 trunc from 0x7a23807 to 0x47cbcd2 2024-09-16T14:07:19.903 INFO:teuthology.orchestra.run.smithi079.stdout:503 read 0x18f7d52 thru 0x1907b59 (0xfe08 bytes) 2024-09-16T14:07:19.924 INFO:teuthology.orchestra.run.smithi079.stdout:505 trunc from 0x47cbcd2 to 0xddceb98 2024-09-16T14:07:19.926 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:19.918+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:19.936 INFO:teuthology.orchestra.run.smithi079.stdout:509 read 0xdb75880 thru 0xdb800ff (0xa880 bytes) 2024-09-16T14:07:19.936 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:19.936 INFO:teuthology.orchestra.run.smithi079.stdout:511 read 0x36d05f9 thru 0x36de2af (0xdcb7 bytes) 2024-09-16T14:07:19.936 INFO:teuthology.orchestra.run.smithi079.stdout:512 read 0x34e2cf2 thru 0x34e8d8c (0x609b bytes) 2024-09-16T14:07:19.936 INFO:teuthology.orchestra.run.smithi079.stdout:516 writesame 0xc1f3e84 thru 0xc20017a data_size 0x3(0xc2f7 bytes) 2024-09-16T14:07:19.963 INFO:teuthology.orchestra.run.smithi079.stdout:517 write 0x71c1ec6 thru 0x71d025e (0xe399 bytes) 2024-09-16T14:07:19.969 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:19.969 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:19.981 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:19.982 INFO:teuthology.orchestra.run.smithi079.stdout:518 write 0x34bfea5 thru 0x34c568b (0x57e7 bytes) 2024-09-16T14:07:19.992 INFO:teuthology.orchestra.run.smithi079.stdout:519 read 0x9042f3c thru 0x9047c6c (0x4d31 bytes) 2024-09-16T14:07:19.992 INFO:teuthology.orchestra.run.smithi079.stdout:520 write 0x5a68a37 thru 0x5a6c738 (0x3d02 bytes) 2024-09-16T14:07:20.017 INFO:teuthology.orchestra.run.smithi079.stdout:521 trunc from 0xddceb98 to 0x5f52dbe 2024-09-16T14:07:20.027 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:20.022+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:20.039 INFO:teuthology.orchestra.run.smithi079.stdout:522 write 0x763e77 thru 0x770bff (0xcd89 bytes) 2024-09-16T14:07:20.060 INFO:teuthology.orchestra.run.smithi079.stdout:523 write 0x7a55337 thru 0x7a5a99c (0x5666 bytes) 2024-09-16T14:07:20.084 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:20.084 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:20.093 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:20.098 INFO:teuthology.orchestra.run.smithi079.stdout:524 write 0xe2e728f thru 0xe2efe6e (0x8be0 bytes) 2024-09-16T14:07:20.107 INFO:teuthology.orchestra.run.smithi079.stdout:526 write 0x3fe080e thru 0x3fe6785 (0x5f78 bytes) 2024-09-16T14:07:20.118 INFO:teuthology.orchestra.run.smithi079.stdout:527 punch from 0xd925175 to 0xd930812, (0xb69d bytes) 2024-09-16T14:07:20.118 INFO:teuthology.orchestra.run.smithi079.stdout:530 read 0x99c7ee8 thru 0x99d6a09 (0xeb22 bytes) 2024-09-16T14:07:20.118 INFO:teuthology.orchestra.run.smithi079.stdout:532 read 0x1093de0 thru 0x10a10c8 (0xd2e9 bytes) 2024-09-16T14:07:20.120 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:20.120 INFO:teuthology.orchestra.run.smithi079.stdout:534 write 0x8749894 thru 0x8753ee3 (0xa650 bytes) 2024-09-16T14:07:20.127 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:20.122+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:20.143 INFO:teuthology.orchestra.run.smithi079.stdout:536 read 0xc310ab9 thru 0xc31dcd5 (0xd21d bytes) 2024-09-16T14:07:20.143 INFO:teuthology.orchestra.run.smithi079.stdout:537 punch from 0x3667fe0 to 0x3675638, (0xd658 bytes) 2024-09-16T14:07:20.143 INFO:teuthology.orchestra.run.smithi079.stdout:539 write 0x3df41a thru 0x3e6a8c (0x7673 bytes) 2024-09-16T14:07:20.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:20.148 INFO:teuthology.orchestra.run.smithi079.stdout:541 read 0x1f03105 thru 0x1f0f0de (0xbfda bytes) 2024-09-16T14:07:20.149 INFO:teuthology.orchestra.run.smithi079.stdout:542 read 0x8f235c thru 0x8faa1a (0x86bf bytes) 2024-09-16T14:07:20.150 INFO:teuthology.orchestra.run.smithi079.stdout:543 write 0xc7055cf thru 0xc70f8db (0xa30d bytes) 2024-09-16T14:07:20.164 INFO:teuthology.orchestra.run.smithi079.stdout:544 write 0x3a6da7d thru 0x3a7bd3d (0xe2c1 bytes) 2024-09-16T14:07:20.182 INFO:teuthology.orchestra.run.smithi079.stdout:545 write 0x811d4ce thru 0x812a94d (0xd480 bytes) 2024-09-16T14:07:20.196 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:20.196 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:20.198 INFO:teuthology.orchestra.run.smithi079.stdout:547 punch from 0x7faa26a to 0x7fb209d, (0x7e33 bytes) 2024-09-16T14:07:20.198 INFO:teuthology.orchestra.run.smithi079.stdout:549 read 0x4ca8f11 thru 0x4cae801 (0x58f1 bytes) 2024-09-16T14:07:20.198 INFO:teuthology.orchestra.run.smithi079.stdout:552 write 0x3f24724 thru 0x3f2c2b6 (0x7b93 bytes) 2024-09-16T14:07:20.206 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:20.227 INFO:teuthology.orchestra.run.smithi079.stdout:553 trunc from 0xe2efe6f to 0x214dc56 2024-09-16T14:07:20.248 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:20.242+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:20.260 INFO:teuthology.orchestra.run.smithi079.stdout:555 read 0x54af03 thru 0x554adc (0x9bda bytes) 2024-09-16T14:07:20.261 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:07:20.261 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:07:20.261 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T14:07:20.261 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.242842 2024-09-16T14:07:20.262 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.24284206366026273 2024-09-16T14:07:20.285 INFO:teuthology.orchestra.run.smithi079.stdout:557 write 0xbb13063 thru 0xbb1a4c8 (0x7466 bytes) 2024-09-16T14:07:20.296 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:20.301 INFO:teuthology.orchestra.run.smithi079.stdout:559 write 0x4c700a thru 0x4d17a7 (0xa79e bytes) 2024-09-16T14:07:20.305 INFO:teuthology.orchestra.run.smithi079.stdout:560 punch from 0x723e82a to 0x724d2e1, (0xeab7 bytes) 2024-09-16T14:07:20.305 INFO:teuthology.orchestra.run.smithi079.stdout:561 read 0x4578629 thru 0x457fb93 (0x756b bytes) 2024-09-16T14:07:20.305 INFO:teuthology.orchestra.run.smithi079.stdout:562 punch from 0x5c0d1ad to 0x5c19f44, (0xcd97 bytes) 2024-09-16T14:07:20.305 INFO:teuthology.orchestra.run.smithi079.stdout:564 write 0xa93dd4 thru 0xa9e975 (0xaba2 bytes) 2024-09-16T14:07:20.315 INFO:teuthology.orchestra.run.smithi079.stdout:565 punch from 0x79b5220 to 0x79b96fa, (0x44da bytes) 2024-09-16T14:07:20.315 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:20.315 INFO:teuthology.orchestra.run.smithi079.stdout:567 read 0x97236cd thru 0x972909d (0x59d1 bytes) 2024-09-16T14:07:20.315 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:20.315 INFO:teuthology.orchestra.run.smithi079.stdout:570 read 0x7f6aad3 thru 0x7f6db8f (0x30bd bytes) 2024-09-16T14:07:20.315 INFO:teuthology.orchestra.run.smithi079.stdout:571 read 0x6fd7ffd thru 0x6fdafd3 (0x2fd7 bytes) 2024-09-16T14:07:20.316 INFO:teuthology.orchestra.run.smithi079.stdout:572 write 0x5529be1 thru 0x552b135 (0x1555 bytes) 2024-09-16T14:07:20.318 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:20.320 INFO:teuthology.orchestra.run.smithi079.stdout:573 read 0x7678cab thru 0x767d458 (0x47ae bytes) 2024-09-16T14:07:20.320 INFO:teuthology.orchestra.run.smithi079.stdout:576 read 0x6faab1c thru 0x6fb9498 (0xe97d bytes) 2024-09-16T14:07:20.326 INFO:teuthology.orchestra.run.smithi079.stdout:577 trunc from 0xbb1a4c9 to 0x83eb897 2024-09-16T14:07:20.328 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:20.322+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:20.342 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:20.343 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:20.349 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:20.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:20.355 INFO:teuthology.orchestra.run.smithi079.stdout:579 write 0x4597774 thru 0x459dd13 (0x65a0 bytes) 2024-09-16T14:07:20.370 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:20.375 INFO:teuthology.orchestra.run.smithi079.stdout:581 write 0x851e38b thru 0x8521d45 (0x39bb bytes) 2024-09-16T14:07:20.386 INFO:teuthology.orchestra.run.smithi079.stdout:582 write 0xe16ca8 thru 0xe19818 (0x2b71 bytes) 2024-09-16T14:07:20.390 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:20.391 INFO:teuthology.orchestra.run.smithi079.stdout:584 trunc from 0x8521d46 to 0x63a4b66 2024-09-16T14:07:20.416 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:20.417 INFO:teuthology.orchestra.run.smithi079.stdout:585 trunc from 0x63a4b66 to 0xaf644fc 2024-09-16T14:07:20.426 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:20.429 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:20.422+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:20.438 INFO:teuthology.orchestra.run.smithi079.stdout:587 trunc from 0xaf644fc to 0x553740e 2024-09-16T14:07:20.438 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:20.448 INFO:teuthology.orchestra.run.smithi079.stdout:588 read 0x3070eb3 thru 0x3074172 (0x32c0 bytes) 2024-09-16T14:07:20.461 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:20.462 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:20.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:20.465 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:20.465 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:20.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:884 3.12 3:4de31fa4:::rbd_data.1136e0d3367d.000000000000000a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7495100290000001, 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056685160000000002, 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 884 2024-09-16T14:07:20.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711294+0000", 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7669999999999999e-06 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711309+0000", 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4185999999999999e-05 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711391+0000", 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1997999999999997e-05 2024-09-16T14:07:20.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711408+0000", 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7782999999999999e-05 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711606+0000", 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019735900000000001 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711909+0000", 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030280699999999999 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716960+0000", 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050516160000000001 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.470 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:894 3.1 3:834411d3:::rbd_data.1136e0d3367d.0000000000000011:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7449197920000001, 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067046559999999998, 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 894 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715885+0000", 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9299999999999999e-06 2024-09-16T14:07:20.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715897+0000", 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2099999999999999e-05 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715943+0000", 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5785000000000003e-05 2024-09-16T14:07:20.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715970+0000", 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7529999999999999e-05 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716054+0000", 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4041000000000004e-05 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716135+0000", 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0524000000000004e-05 2024-09-16T14:07:20.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716338+0000", 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000203196 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717189+0000", 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085096599999999998 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:20.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722487+0000", 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052979530000000002 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722541+0000", 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4268999999999998e-05 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722587+0000", 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5361999999999997e-05 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1105 3.1 3:86ae877e:::rbd_data.11365ca8c0d.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.255776+0000", 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.2050259589999999, 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049602739999999998, 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1105 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255776+0000", 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255776+0000", 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255777+0000", 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6020000000000001e-06 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255808+0000", 2024-09-16T14:07:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0493000000000001e-05 2024-09-16T14:07:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255824+0000", 2024-09-16T14:07:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5794000000000002e-05 2024-09-16T14:07:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255833+0000", 2024-09-16T14:07:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9800000000000004e-06 2024-09-16T14:07:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.255922+0000", 2024-09-16T14:07:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9208000000000004e-05 2024-09-16T14:07:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.256157+0000", 2024-09-16T14:07:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023554700000000001 2024-09-16T14:07:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.256535+0000", 2024-09-16T14:07:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00037740000000000001 2024-09-16T14:07:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.257653+0000", 2024-09-16T14:07:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011181870000000001 2024-09-16T14:07:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.260682+0000", 2024-09-16T14:07:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030295769999999999 2024-09-16T14:07:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.260712+0000", 2024-09-16T14:07:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9941e-05 2024-09-16T14:07:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.260736+0000", 2024-09-16T14:07:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3544999999999999e-05 2024-09-16T14:07:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1519 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.871666+0000", 2024-09-16T14:07:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.58913584, 2024-09-16T14:07:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046516190000000001, 2024-09-16T14:07:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1519 2024-09-16T14:07:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.871666+0000", 2024-09-16T14:07:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.871666+0000", 2024-09-16T14:07:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.871666+0000", 2024-09-16T14:07:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2300000000000001e-07 2024-09-16T14:07:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.871670+0000", 2024-09-16T14:07:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4139999999999998e-06 2024-09-16T14:07:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.871677+0000", 2024-09-16T14:07:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1690000000000003e-06 2024-09-16T14:07:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.871683+0000", 2024-09-16T14:07:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8599999999999998e-06 2024-09-16T14:07:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.876032+0000", 2024-09-16T14:07:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043490589999999997 2024-09-16T14:07:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.876162+0000", 2024-09-16T14:07:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012989100000000001 2024-09-16T14:07:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.876317+0000", 2024-09-16T14:07:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015560300000000001 2024-09-16T14:07:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1548 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.872971+0000", 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.587830528, 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046938320000000002, 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1548 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.872971+0000", 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.872971+0000", 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.872972+0000", 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.23e-07 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.872974+0000", 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1600000000000001e-06 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.872984+0000", 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5079999999999994e-06 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.872991+0000", 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.877416+0000", 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044244769999999996 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.877526+0000", 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011053 2024-09-16T14:07:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.877665+0000", 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013871700000000001 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1549 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873000+0000", 2024-09-16T14:07:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.5878015599999999, 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049577859999999996, 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1549 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873000+0000", 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873000+0000", 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873001+0000", 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1999999999999999e-06 2024-09-16T14:07:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873004+0000", 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873012+0000", 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6660000000000008e-06 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873018+0000", 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.699e-06 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.877720+0000", 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047021290000000002 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.877843+0000", 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012262500000000001 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.877958+0000", 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011537 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1550 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873025+0000", 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.587776275, 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051859499999999999, 2024-09-16T14:07:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1550 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873025+0000", 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873025+0000", 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873026+0000", 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873031+0000", 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0170000000000004e-06 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873045+0000", 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3614e-05 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873052+0000", 2024-09-16T14:07:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4309999999999997e-06 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878009+0000", 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049567370000000001 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878100+0000", 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0717999999999994e-05 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878211+0000", 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011165 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1551 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873068+0000", 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.5877332389999999, 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053738780000000003, 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1551 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873068+0000", 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873068+0000", 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873069+0000", 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7999999999999993e-07 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873074+0000", 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6630000000000001e-06 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873088+0000", 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3617e-05 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873096+0000", 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9570000000000002e-06 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878262+0000", 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005166187 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878348+0000", 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6055000000000002e-05 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878442+0000", 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4419000000000005e-05 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1577 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873390+0000", 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.5874112569999999, 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005292265, 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1577 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873390+0000", 2024-09-16T14:07:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873390+0000", 2024-09-16T14:07:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873392+0000", 2024-09-16T14:07:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7570000000000001e-06 2024-09-16T14:07:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873402+0000", 2024-09-16T14:07:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0271e-05 2024-09-16T14:07:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873424+0000", 2024-09-16T14:07:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1797000000000002e-05 2024-09-16T14:07:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873438+0000", 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3805000000000001e-05 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878488+0000", 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050503370000000002 2024-09-16T14:07:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878571+0000", 2024-09-16T14:07:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2327e-05 2024-09-16T14:07:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878683+0000", 2024-09-16T14:07:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111971 2024-09-16T14:07:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1578 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873458+0000", 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.5873435300000001, 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054174619999999996, 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1578 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873458+0000", 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873458+0000", 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873459+0000", 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.189e-06 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873466+0000", 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2020000000000001e-06 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873475+0000", 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9340000000000001e-06 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873482+0000", 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4489999999999996e-06 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878722+0000", 2024-09-16T14:07:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052398369999999998 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878791+0000", 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8814999999999998e-05 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878876+0000", 2024-09-16T14:07:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5036000000000004e-05 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1579 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873535+0000", 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.5872663170000001, 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055303089999999997, 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1579 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873535+0000", 2024-09-16T14:07:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873535+0000", 2024-09-16T14:07:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873536+0000", 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.06e-06 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873540+0000", 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873550+0000", 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0811e-05 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873556+0000", 2024-09-16T14:07:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3000000000000001e-06 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878915+0000", 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053588769999999997 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878983+0000", 2024-09-16T14:07:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8497000000000005e-05 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879066+0000", 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2615999999999996e-05 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1580 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873561+0000", 2024-09-16T14:07:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.5872407009999998, 2024-09-16T14:07:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005697967, 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1580 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873561+0000", 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873561+0000", 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873562+0000", 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.09e-07 2024-09-16T14:07:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873565+0000", 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9050000000000001e-06 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873572+0000", 2024-09-16T14:07:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4200000000000001e-06 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873580+0000", 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4970000000000004e-06 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879104+0000", 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055246569999999997 2024-09-16T14:07:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879172+0000", 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7815e-05 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879259+0000", 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6763999999999998e-05 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1581 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873585+0000", 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.58721661, 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058658360000000001, 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1581 2024-09-16T14:07:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873585+0000", 2024-09-16T14:07:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873585+0000", 2024-09-16T14:07:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873586+0000", 2024-09-16T14:07:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3399999999999999e-07 2024-09-16T14:07:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873588+0000", 2024-09-16T14:07:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4949999999999998e-06 2024-09-16T14:07:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873595+0000", 2024-09-16T14:07:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0280000000000003e-06 2024-09-16T14:07:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873601+0000", 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879298+0000", 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056969430000000003 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879368+0000", 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0217999999999998e-05 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879451+0000", 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2800999999999995e-05 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1582 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873606+0000", 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.5871956680000001, 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006036563, 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1582 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873606+0000", 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873606+0000", 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873607+0000", 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2300000000000001e-07 2024-09-16T14:07:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873609+0000", 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5000000000000002e-06 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873616+0000", 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873621+0000", 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3170000000000003e-06 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879489+0000", 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058687239999999996 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879559+0000", 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9270000000000006e-05 2024-09-16T14:07:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879643+0000", 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4046000000000001e-05 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1583 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873626+0000", 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.5871761279999999, 2024-09-16T14:07:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061920619999999999, 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1583 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873626+0000", 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873626+0000", 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873626+0000", 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2600000000000002e-07 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873629+0000", 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.225e-06 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873637+0000", 2024-09-16T14:07:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2289999999999997e-06 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873641+0000", 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3649999999999997e-06 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879667+0000", 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060265120000000004 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879736+0000", 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8127999999999996e-05 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879818+0000", 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1977000000000005e-05 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1768 3.1 3:834411d3:::rbd_data.1136e0d3367d.0000000000000011:5 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.019124+0000", 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.4416773460000001, 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045207800000000003, 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1768 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019124+0000", 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019124+0000", 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019125+0000", 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.82e-07 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019127+0000", 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2350000000000002e-06 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019133+0000", 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-06 2024-09-16T14:07:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019138+0000", 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9520000000000002e-06 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.023459+0000", 2024-09-16T14:07:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043215079999999999 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.023559+0000", 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9845000000000004e-05 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.023645+0000", 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5978999999999998e-05 2024-09-16T14:07:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1772 3.1 3:834411d3:::rbd_data.1136e0d3367d.0000000000000011:5 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.019302+0000", 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.441499393, 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046059040000000001, 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1772 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019302+0000", 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019302+0000", 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019304+0000", 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019308+0000", 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2930000000000002e-06 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019318+0000", 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6250000000000002e-06 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019324+0000", 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5409999999999997e-06 2024-09-16T14:07:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.023679+0000", 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043547309999999997 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.023775+0000", 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6082999999999995e-05 2024-09-16T14:07:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.023908+0000", 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013326399999999999 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1776 3.1 3:834411d3:::rbd_data.1136e0d3367d.0000000000000011:5 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.019329+0000", 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.4414724400000001, 2024-09-16T14:07:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048620920000000001, 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1776 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019329+0000", 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019329+0000", 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019330+0000", 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2200000000000004e-07 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019332+0000", 2024-09-16T14:07:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4090000000000001e-06 2024-09-16T14:07:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019341+0000", 2024-09-16T14:07:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3930000000000008e-06 2024-09-16T14:07:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019346+0000", 2024-09-16T14:07:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0690000000000001e-06 2024-09-16T14:07:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.023950+0000", 2024-09-16T14:07:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046040999999999999 2024-09-16T14:07:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.024063+0000", 2024-09-16T14:07:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113694 2024-09-16T14:07:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.024191+0000", 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012790499999999999 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1782 3.1 3:834411d3:::rbd_data.1136e0d3367d.0000000000000011:5 [read 466944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.019350+0000", 2024-09-16T14:07:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.4414520799999999, 2024-09-16T14:07:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052180259999999997, 2024-09-16T14:07:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1782 2024-09-16T14:07:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019350+0000", 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019350+0000", 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019350+0000", 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:07:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019352+0000", 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9290000000000001e-06 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019360+0000", 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9689999999999994e-06 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.019364+0000", 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2780000000000004e-06 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.024371+0000", 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050069750000000003 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.024466+0000", 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4338000000000004e-05 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.024568+0000", 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102062 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1834 3.5 e31/23 3:a4605f64:::rbd_data.1136494ba971.00000000000001fa:head v 31'19, mlcod=31'15)", 2024-09-16T14:07:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.077251+0000", 2024-09-16T14:07:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.3835502650000002, 2024-09-16T14:07:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056664009999999997, 2024-09-16T14:07:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077251+0000", 2024-09-16T14:07:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077251+0000", 2024-09-16T14:07:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077252+0000", 2024-09-16T14:07:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.02e-06 2024-09-16T14:07:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077261+0000", 2024-09-16T14:07:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0650000000000005e-06 2024-09-16T14:07:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077276+0000", 2024-09-16T14:07:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4871e-05 2024-09-16T14:07:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077285+0000", 2024-09-16T14:07:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4270000000000008e-06 2024-09-16T14:07:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077360+0000", 2024-09-16T14:07:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4826999999999994e-05 2024-09-16T14:07:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077380+0000", 2024-09-16T14:07:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.039e-05 2024-09-16T14:07:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.082847+0000", 2024-09-16T14:07:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054671470000000003 2024-09-16T14:07:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.082918+0000", 2024-09-16T14:07:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0654000000000005e-05 2024-09-16T14:07:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.594 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:20.596 INFO:teuthology.orchestra.run.smithi079.stdout:589 write 0xa510f13 thru 0xa51b6d3 (0xa7c1 bytes) 2024-09-16T14:07:20.597 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:20.598 INFO:teuthology.orchestra.run.smithi079.stdout:593 punch from 0x96b027 to 0x977790, (0xc769 bytes) 2024-09-16T14:07:20.598 INFO:teuthology.orchestra.run.smithi079.stdout:truncating to largest ever: 0xe380509 2024-09-16T14:07:20.598 INFO:teuthology.orchestra.run.smithi079.stdout:596 trunc from 0xa51b6d4 to 0xe380509 2024-09-16T14:07:20.598 INFO:teuthology.orchestra.run.smithi079.stdout:597 clone 4 order 23 su 65536 sc 13 2024-09-16T14:07:20.598 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:20.590+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:20.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:07:20.697 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:20.690+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:20.707 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:20.707 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:20.707 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:20.707 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:20.716 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:20.798 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:20.790+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:20.820 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:20.821 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:20.821 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:20.830 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:20.899 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:20.894+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:20.938 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:20.938 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:20.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:20.938 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:20.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.939 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:13 3.8 e26/23 3:14a6dda4:::rbd_object_map.113631a28a63:head v 26'1, mlcod=0'0)", 2024-09-16T14:07:20.939 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:11.368040+0000", 2024-09-16T14:07:20.939 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.5642900419999997, 2024-09-16T14:07:20.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038663970000000001, 2024-09-16T14:07:20.939 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.939 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368040+0000", 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368040+0000", 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368041+0000", 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2699999999999999e-07 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368052+0000", 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0879999999999999e-05 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368098+0000", 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5429000000000002e-05 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368113+0000", 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5016e-05 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368215+0000", 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102394 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.368262+0000", 2024-09-16T14:07:20.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7453000000000001e-05 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.371771+0000", 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035084270000000002 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:11.371907+0000", 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013607100000000001 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:735 3.6 e29/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 29'108, mlcod=29'106)", 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.853455+0000", 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0788755099999996, 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026461089999999998, 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853455+0000", 2024-09-16T14:07:20.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853455+0000", 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853457+0000", 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6130000000000001e-06 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853469+0000", 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.219e-05 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853500+0000", 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0929000000000002e-05 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853511+0000", 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1283e-05 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853590+0000", 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9419000000000007e-05 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.853647+0000", 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6335999999999997e-05 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.855948+0000", 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002301619 2024-09-16T14:07:20.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.856101+0000", 2024-09-16T14:07:20.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015271999999999999 2024-09-16T14:07:20.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.949 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:749 3.b 3:d49083f7:::rbd_data.1136e0d3367d.00000000000001a3:head [set-alloc-hint object_size 524288 write_size 524288,write 87906~8504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.890836+0000", 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0414944090000002, 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002955226, 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.950 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 749 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890836+0000", 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890836+0000", 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890839+0000", 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8229999999999999e-06 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890865+0000", 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6152999999999998e-05 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890898+0000", 2024-09-16T14:07:20.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2879000000000001e-05 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890911+0000", 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2773e-05 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.890993+0000", 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1773000000000005e-05 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891212+0000", 2024-09-16T14:07:20.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000219185 2024-09-16T14:07:20.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:20.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.891469+0000", 2024-09-16T14:07:20.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025680599999999999 2024-09-16T14:07:20.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.892862+0000", 2024-09-16T14:07:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013933120000000001 2024-09-16T14:07:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:20.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.893586+0000", 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072398000000000004 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.893680+0000", 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4565e-05 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.893791+0000", 2024-09-16T14:07:20.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110977 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:751 3.b 3:d49083f7:::rbd_data.1136e0d3367d.00000000000001a3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.901387+0000", 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.0309432010000004, 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028494919999999999, 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.957 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 751 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901387+0000", 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901387+0000", 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901389+0000", 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0940000000000002e-06 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901401+0000", 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1613e-05 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901436+0000", 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4511999999999998e-05 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901467+0000", 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1856999999999997e-05 2024-09-16T14:07:20.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901583+0000", 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115383 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.901759+0000", 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000176393 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:20.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.902011+0000", 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000252134 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.903267+0000", 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001255548 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.903974+0000", 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070696300000000001 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.904119+0000", 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014561399999999999 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.904237+0000", 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011738100000000001 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.963 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:782 3.6 e29/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 29'118, mlcod=29'116)", 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.971080+0000", 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.9612501959999999, 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024601340000000001, 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971080+0000", 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971080+0000", 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971082+0000", 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.604e-06 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971095+0000", 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.259e-05 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971126+0000", 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1282000000000003e-05 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971137+0000", 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270000000000001e-05 2024-09-16T14:07:20.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971229+0000", 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1509999999999996e-05 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.971280+0000", 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1799000000000003e-05 2024-09-16T14:07:20.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.973449+0000", 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00216898 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.973540+0000", 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1099000000000003e-05 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:791 3.6 e29/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 29'124, mlcod=29'121)", 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:15.988609+0000", 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.9437215200000004, 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026732599999999998, 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988609+0000", 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988609+0000", 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988610+0000", 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1100000000000005e-07 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988618+0000", 2024-09-16T14:07:20.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5969999999999992e-06 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988636+0000", 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7510999999999999e-05 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988643+0000", 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7880000000000001e-06 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988720+0000", 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7059999999999997e-05 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.988756+0000", 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5760999999999999e-05 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.991240+0000", 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002484109 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:15.991282+0000", 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2623000000000002e-05 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:880 3.6 e30/23 3:65cfeb1a:::rbd_data.1136e0d3367d.0000000000000006:head v 30'128, mlcod=30'125)", 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.712596+0000", 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2197349470000001, 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025605939999999998, 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712596+0000", 2024-09-16T14:07:20.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712596+0000", 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712597+0000", 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.823e-06 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712615+0000", 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7706e-05 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712664+0000", 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9388999999999998e-05 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712681+0000", 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6569999999999999e-05 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712926+0000", 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024498100000000002 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.712982+0000", 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5628999999999999e-05 2024-09-16T14:07:20.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715011+0000", 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020292819999999999 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715156+0000", 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014521400000000001 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:907 3.6 e30/23 3:6479ea17:::rbd_data.1136e0d3367d.000000000000008f:head v 30'130, mlcod=30'126)", 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.723354+0000", 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2089762779999997, 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023499089999999999, 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723354+0000", 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723354+0000", 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723356+0000", 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.818e-06 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723369+0000", 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2466000000000001e-05 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723391+0000", 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2238999999999998e-05 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723402+0000", 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0976e-05 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723471+0000", 2024-09-16T14:07:20.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9183000000000005e-05 2024-09-16T14:07:20.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.723496+0000", 2024-09-16T14:07:20.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5559e-05 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.725644+0000", 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021471820000000001 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.725704+0000", 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0486000000000003e-05 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.984 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:918 3.6 e30/23 3:66edd6af:::rbd_object_map.1136e0d3367d:head v 30'137, mlcod=30'134)", 2024-09-16T14:07:20.984 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.740936+0000", 2024-09-16T14:07:20.984 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1913943959999997, 2024-09-16T14:07:20.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029995500000000001, 2024-09-16T14:07:20.984 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.984 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.984 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740936+0000", 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740936+0000", 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740937+0000", 2024-09-16T14:07:20.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.06e-06 2024-09-16T14:07:20.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740953+0000", 2024-09-16T14:07:20.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5829e-05 2024-09-16T14:07:20.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740969+0000", 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6226e-05 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740981+0000", 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.201e-05 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.741092+0000", 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111093 2024-09-16T14:07:20.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.741131+0000", 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8383999999999997e-05 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.743890+0000", 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027588719999999999 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.743936+0000", 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1062 3.b 3:d2620549:::rbd_data.11365ca8c0d.000000000000006e:head [set-alloc-hint object_size 2097152 write_size 2097152,write 343358~60766] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:20.989 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.205224+0000", 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.727106166, 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003075225, 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1062 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205224+0000", 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205224+0000", 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205227+0000", 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4779999999999998e-06 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205430+0000", 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020357799999999999 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205468+0000", 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7493000000000002e-05 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205491+0000", 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2722e-05 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205579+0000", 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8214999999999995e-05 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.205784+0000", 2024-09-16T14:07:20.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020552499999999999 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.206006+0000", 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022118100000000001 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.207753+0000", 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017474260000000001 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:20.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.208145+0000", 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000391956 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.208229+0000", 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4080999999999997e-05 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:20.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.208300+0000", 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0569999999999997e-05 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1068 3.b 3:d3284eff:::rbd_data.11365ca8c0d.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.233409+0000", 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.6989218089999998, 2024-09-16T14:07:20.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042937849999999996, 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1068 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233409+0000", 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233409+0000", 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233412+0000", 2024-09-16T14:07:20.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2600000000000001e-06 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233429+0000", 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7146000000000002e-05 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233474+0000", 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5346999999999998e-05 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:20.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233493+0000", 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8051e-05 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233684+0000", 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019169900000000001 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233921+0000", 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000236598 2024-09-16T14:07:21.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:21.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234247+0000", 2024-09-16T14:07:21.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000326034 2024-09-16T14:07:21.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:21.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.236457+0000", 2024-09-16T14:07:21.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002209866 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.237465+0000", 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010083659999999999 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.237596+0000", 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130903 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.237702+0000", 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010651500000000001 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1072 3.b 3:d2620549:::rbd_data.11365ca8c0d.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.233527+0000", 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.698803737, 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048508839999999998, 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1072 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233527+0000", 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233527+0000", 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233529+0000", 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233536+0000", 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0879999999999997e-06 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233565+0000", 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8906000000000001e-05 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.233576+0000", 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1439e-05 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234757+0000", 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011812750000000001 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.234949+0000", 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000191929 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:21.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.235253+0000", 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030329400000000001 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.237056+0000", 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018033260000000001 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.238262+0000", 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00120623 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.238327+0000", 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4457999999999998e-05 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.238378+0000", 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1078999999999997e-05 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1165 3.8 e30/23 3:158a6710:::rbd_data.11365ca8c0d.000000000000001c:head v 30'36, mlcod=30'34)", 2024-09-16T14:07:21.010 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.356578+0000", 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.575752058, 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034399999999999999, 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356578+0000", 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356578+0000", 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356580+0000", 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8360000000000001e-06 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356600+0000", 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9324e-05 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356634+0000", 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4756999999999999e-05 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356650+0000", 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5159e-05 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356719+0000", 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9188000000000002e-05 2024-09-16T14:07:21.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356744+0000", 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5626000000000002e-05 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.359957+0000", 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032128159999999998 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360018+0000", 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1294e-05 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1167 3.6 e30/23 3:637d63d5:::rbd_data.11365ca8c0d.000000000000001d:head v 30'141, mlcod=30'139)", 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.358955+0000", 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.5733753539999999, 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035207749999999999, 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.015 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358955+0000", 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358955+0000", 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358956+0000", 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1299999999999998e-07 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358964+0000", 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5839999999999997e-06 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358980+0000", 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6118000000000001e-05 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358988+0000", 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4370000000000007e-06 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.359080+0000", 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2120000000000006e-05 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.359130+0000", 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0058000000000001e-05 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362401+0000", 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003270991 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362476+0000", 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4554000000000005e-05 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.019 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1323 3.b 3:d48dcf62:::rbd_data.11365ca8c0d.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.719615+0000", 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.2127155439999999, 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043393909999999997, 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:21.020 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1323 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719615+0000", 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719615+0000", 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719618+0000", 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3189999999999999e-06 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719630+0000", 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1762e-05 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719662+0000", 2024-09-16T14:07:21.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1634999999999998e-05 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719673+0000", 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1759e-05 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719806+0000", 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013287300000000001 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:07:21.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719815+0000", 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439999999999992e-06 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.722497+0000", 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026811220000000002 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.722610+0000", 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113792 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.722771+0000", 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016077299999999999 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.723475+0000", 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070413099999999998 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.723861+0000", 2024-09-16T14:07:21.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038613100000000001 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.723903+0000", 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1505000000000003e-05 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.723954+0000", 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1445e-05 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1330 3.c e31/23 3:35e9eee3:::rbd_data.11365ca8c0d.0000000000000000:head v 31'8, mlcod=29'5)", 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.725914+0000", 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.206416333, 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021855289999999999, 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.725914+0000", 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.725914+0000", 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.725914+0000", 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9299999999999999e-07 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.725919+0000", 2024-09-16T14:07:21.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7670000000000003e-06 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.725928+0000", 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8999999999999995e-06 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.725933+0000", 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5770000000000004e-06 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.725995+0000", 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1866999999999996e-05 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.726012+0000", 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7847e-05 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728052+0000", 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020398790000000001 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728100+0000", 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7398999999999998e-05 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1619 3.0 e31/23 3:0374e4d4:::rbd_data.1136494ba971.0000000000000204:head v 31'12, mlcod=31'9)", 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.940797+0000", 2024-09-16T14:07:21.031 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.991533241, 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031332819999999998, 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.940797+0000", 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.940797+0000", 2024-09-16T14:07:21.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.940800+0000", 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.526e-06 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.940814+0000", 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3919e-05 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.940851+0000", 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7453999999999997e-05 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.940866+0000", 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4491000000000001e-05 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.940960+0000", 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3840999999999998e-05 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.940997+0000", 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7364999999999999e-05 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.943874+0000", 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028775509999999999 2024-09-16T14:07:21.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.943931+0000", 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6134999999999997e-05 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1670 3.0 e31/23 3:0644b61e:::rbd_data.1136494ba971.00000000000001fc:head v 31'14, mlcod=31'12)", 2024-09-16T14:07:21.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.011645+0000", 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.92068584, 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002516683, 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011645+0000", 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011645+0000", 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011647+0000", 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3939999999999999e-06 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011679+0000", 2024-09-16T14:07:21.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2021999999999999e-05 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011720+0000", 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0887999999999999e-05 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011737+0000", 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7388e-05 2024-09-16T14:07:21.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011918+0000", 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000180972 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011971+0000", 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979000000000002e-05 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.014100+0000", 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021287979999999999 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.014161+0000", 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1241999999999994e-05 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1836 3.0 e31/23 3:0644b61e:::rbd_data.1136494ba971.00000000000001fc:head v 31'16, mlcod=31'14)", 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.076737+0000", 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.8555939509999999, 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031132719999999998, 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.076737+0000", 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.076737+0000", 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.076738+0000", 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6899999999999996e-07 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.076747+0000", 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2220000000000008e-06 2024-09-16T14:07:21.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.076768+0000", 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0877999999999999e-05 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.076781+0000", 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3098999999999999e-05 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.076886+0000", 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105407 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.076921+0000", 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4811999999999998e-05 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.079753+0000", 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028315990000000002 2024-09-16T14:07:21.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.079850+0000", 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3901 3.b 3:d2c5070a:::rbd_data.1136494ba971.0000000000000294:head [set-alloc-hint object_size 262144 write_size 262144,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:20.470878+0000", 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.46145299400000001, 2024-09-16T14:07:21.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026024199999999998, 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3901 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.470878+0000", 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.470878+0000", 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.470882+0000", 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.98e-06 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.470911+0000", 2024-09-16T14:07:21.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9243000000000001e-05 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.470954+0000", 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3387000000000003e-05 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.470971+0000", 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7099999999999999e-05 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.471102+0000", 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131218 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.471408+0000", 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030574900000000002 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.471701+0000", 2024-09-16T14:07:21.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000293102 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.472934+0000", 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012328759999999999 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473246+0000", 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000311422 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473377+0000", 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131574 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473480+0000", 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102769 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:21.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:21.053 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:21.054 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:21.046+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:21.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:21.155 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:21.150+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:21.165 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:21.165 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:21.165 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:21.165 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:21.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:21.257 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:21.250+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:21.278 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:21.278 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:21.278 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:21.278 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:21.278 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:21.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:21.353 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:07:21.353 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.4 primary-affinity to 0.242842 (8159142) 2024-09-16T14:07:21.357 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:21.350+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:21.404 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:21.404 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:21.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:21.404 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:21.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.404 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-16T14:07:21.404 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.348446+0000", 2024-09-16T14:07:21.404 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.049706938, 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045895969999999999, 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348446+0000", 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348446+0000", 2024-09-16T14:07:21.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348446+0000", 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.77e-07 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348452+0000", 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9440000000000002e-06 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348477+0000", 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4967e-05 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348486+0000", 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.799e-06 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348561+0000", 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5389999999999995e-05 2024-09-16T14:07:21.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.348595+0000", 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4235e-05 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.352954+0000", 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043584519999999996 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.353035+0000", 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1433000000000005e-05 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:678 3.15 3:a91832ea:::rbd_data.1136e0d3367d.0000000000000051:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.725062+0000", 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.6730909369999996, 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011491754, 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 678 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.725062+0000", 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.725062+0000", 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.725064+0000", 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.632e-06 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.725075+0000", 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1143e-05 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.725106+0000", 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1183999999999998e-05 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.725118+0000", 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1714e-05 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.725189+0000", 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0450999999999993e-05 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.725380+0000", 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019092600000000001 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.727111+0000", 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001731233 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.736372+0000", 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092613420000000005 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.736393+0000", 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0733e-05 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.736482+0000", 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8646000000000005e-05 2024-09-16T14:07:21.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.736553+0000", 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1749999999999996e-05 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:737 3.f 3:f02dbe32:::rbd_data.1136e0d3367d.0000000000000064:head [set-alloc-hint object_size 524288 write_size 524288,write 173391~23217] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.857871+0000", 2024-09-16T14:07:21.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.5402814129999998, 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038675390000000001, 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 737 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.857871+0000", 2024-09-16T14:07:21.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.857871+0000", 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.857874+0000", 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9399999999999998e-06 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.857905+0000", 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0565000000000003e-05 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.857942+0000", 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7459000000000002e-05 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.857955+0000", 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2751e-05 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858029+0000", 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3897000000000001e-05 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858120+0000", 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1583999999999996e-05 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.858301+0000", 2024-09-16T14:07:21.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018047500000000001 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.859761+0000", 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014597939999999999 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.861628+0000", 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018670710000000001 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.861690+0000", 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2325999999999999e-05 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.861739+0000", 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8677000000000002e-05 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:781 3.15 3:ac311a3e:::rbd_data.1136e0d3367d.000000000000003f:head [set-alloc-hint object_size 524288 write_size 524288,write 32951~27414] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.966605+0000", 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4315478180000003, 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036986129999999999, 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 781 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.966605+0000", 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.966605+0000", 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.966608+0000", 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.016e-06 2024-09-16T14:07:21.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.966643+0000", 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5586000000000001e-05 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.966686+0000", 2024-09-16T14:07:21.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.244e-05 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.966706+0000", 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0006000000000001e-05 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.966822+0000", 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116001 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.967078+0000", 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025660400000000001 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.967330+0000", 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000251315 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.968906+0000", 2024-09-16T14:07:21.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015766389999999999 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.970142+0000", 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001235442 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.970242+0000", 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100222 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.970303+0000", 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1341999999999996e-05 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:862 3.f 3:f699bf02:::rbd_header.11365ca8c0d:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.682937+0000", 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7152158660000003, 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045073869999999999, 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 862 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.682937+0000", 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.682937+0000", 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.682939+0000", 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1110000000000002e-06 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.682948+0000", 2024-09-16T14:07:21.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6399999999999992e-06 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.682971+0000", 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3099e-05 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.682995+0000", 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3597000000000001e-05 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.683067+0000", 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1447999999999998e-05 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.683172+0000", 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010561299999999999 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.683481+0000", 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030888399999999999 2024-09-16T14:07:21.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.687235+0000", 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037539050000000001 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.687359+0000", 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012372199999999999 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.687410+0000", 2024-09-16T14:07:21.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1224999999999998e-05 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.687444+0000", 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4143000000000002e-05 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:903 3.15 3:a91832ea:::rbd_data.1136e0d3367d.0000000000000051:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.724084+0000", 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6740689179999997, 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037510619999999999, 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 903 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724084+0000", 2024-09-16T14:07:21.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724084+0000", 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724085+0000", 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.035e-06 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724089+0000", 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0929999999999996e-06 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724098+0000", 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4029999999999996e-06 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724117+0000", 2024-09-16T14:07:21.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8706999999999999e-05 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.725321+0000", 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012045140000000001 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.725390+0000", 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8268000000000004e-05 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:21.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.725557+0000", 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016716600000000001 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727081+0000", 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001524547 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727743+0000", 2024-09-16T14:07:21.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066137800000000001 2024-09-16T14:07:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727789+0000", 2024-09-16T14:07:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6689e-05 2024-09-16T14:07:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727835+0000", 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5262000000000001e-05 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:904 3.f 3:f02dbe32:::rbd_data.1136e0d3367d.0000000000000064:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.724125+0000", 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.674027208, 2024-09-16T14:07:21.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044027529999999997, 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 904 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724125+0000", 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724125+0000", 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724126+0000", 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8200000000000008e-07 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724130+0000", 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4620000000000001e-06 2024-09-16T14:07:21.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724139+0000", 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0569999999999999e-06 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724144+0000", 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1449999999999999e-06 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724347+0000", 2024-09-16T14:07:21.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020301299999999999 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.725620+0000", 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012727789999999999 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.725953+0000", 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033327900000000002 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.728289+0000", 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023357299999999998 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.728433+0000", 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014438400000000001 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.728487+0000", 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3677000000000001e-05 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.728528+0000", 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1245000000000002e-05 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:908 3.c 3:33261b8d:::rbd_data.1136e0d3367d.00000000000000d8:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.724163+0000", 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6739898530000001, 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036672190000000002, 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 908 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724163+0000", 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724163+0000", 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724163+0000", 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6199999999999997e-07 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724167+0000", 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3069999999999998e-06 2024-09-16T14:07:21.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724176+0000", 2024-09-16T14:07:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8470000000000002e-06 2024-09-16T14:07:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724181+0000", 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2820000000000004e-06 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724251+0000", 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9873e-05 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724376+0000", 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000125656 2024-09-16T14:07:21.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724882+0000", 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050547299999999999 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.726279+0000", 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001397377 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727737+0000", 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014575600000000001 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727786+0000", 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9681000000000001e-05 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727830+0000", 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3401e-05 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:911 3.f 3:f2db6741:::rbd_data.1136e0d3367d.00000000000000f7:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.724578+0000", 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6735744920000002, 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048689750000000002, 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 911 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.454 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724578+0000", 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724578+0000", 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724579+0000", 2024-09-16T14:07:21.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.057e-06 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724585+0000", 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724621+0000", 2024-09-16T14:07:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6291999999999997e-05 2024-09-16T14:07:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.724629+0000", 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3680000000000002e-06 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727339+0000", 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002709735 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727400+0000", 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0927000000000001e-05 2024-09-16T14:07:21.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.727529+0000", 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000128892 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.729300+0000", 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017716100000000001 2024-09-16T14:07:21.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.729375+0000", 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4797999999999998e-05 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.729415+0000", 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9743000000000002e-05 2024-09-16T14:07:21.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.729447+0000", 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2007e-05 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1021 3.15 3:add72617:::rbd_data.11365ca8c0d.0000000000000035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.155878+0000", 2024-09-16T14:07:21.461 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2422745529999997, 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038143740000000001, 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1021 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155878+0000", 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155878+0000", 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155879+0000", 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3769999999999999e-06 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155887+0000", 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4109999999999999e-06 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155924+0000", 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7662e-05 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155937+0000", 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2549e-05 2024-09-16T14:07:21.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156055+0000", 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118361 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156403+0000", 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000347735 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.159692+0000", 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003289279 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1022 3.15 3:ad0ea9cf:::rbd_data.11365ca8c0d.0000000000000036:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.155964+0000", 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.2421886459999998, 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055692600000000004, 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:21.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1022 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155964+0000", 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155964+0000", 2024-09-16T14:07:21.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155965+0000", 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034e-06 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155970+0000", 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9139999999999999e-06 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155990+0000", 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9754000000000001e-05 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.155999+0000", 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4939999999999992e-06 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156987+0000", 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098750899999999991 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.157261+0000", 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027467099999999999 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.161533+0000", 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042718840000000001 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1096 3.3 3:c57d1a82:::rbd_data.11365ca8c0d.0000000000000016:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.248445+0000", 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.1497079579999996, 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046702970000000003, 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.471 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1096 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.248445+0000", 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.248445+0000", 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.248446+0000", 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2100000000000001e-06 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.248452+0000", 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1720000000000004e-06 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.248467+0000", 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5366000000000002e-05 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.248476+0000", 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7639999999999994e-06 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.248547+0000", 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1155999999999994e-05 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.248791+0000", 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024346799999999999 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.249145+0000", 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035388500000000002 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.252221+0000", 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030765889999999998 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.252931+0000", 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070992500000000003 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.253038+0000", 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010699700000000001 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.253115+0000", 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6765000000000001e-05 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1088 3.1b e30/23 3:de203a46:::rbd_object_map.11365ca8c0d:head v 30'26, mlcod=30'21)", 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.249904+0000", 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.1482484160000004, 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036195519999999998, 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.249904+0000", 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.249904+0000", 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.249905+0000", 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.093e-06 2024-09-16T14:07:21.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.249914+0000", 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6780000000000006e-06 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.249937+0000", 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3663e-05 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.249945+0000", 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.498e-06 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.250038+0000", 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3227999999999996e-05 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.250129+0000", 2024-09-16T14:07:21.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1256999999999997e-05 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.253439+0000", 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033091560000000002 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.253524+0000", 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4979000000000001e-05 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1172 3.1b e30/23 3:de203a46:::rbd_object_map.11365ca8c0d:head v 30'62, mlcod=30'55)", 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.369076+0000", 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0290769409999996, 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047854680000000002, 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.369076+0000", 2024-09-16T14:07:21.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.369076+0000", 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.369076+0000", 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2399999999999998e-07 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.369080+0000", 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0620000000000002e-06 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.369092+0000", 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1987000000000001e-05 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.369097+0000", 2024-09-16T14:07:21.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3970000000000004e-06 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.369126+0000", 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9352999999999998e-05 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.369141+0000", 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4983999999999999e-05 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.373826+0000", 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046846570000000001 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.373861+0000", 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5404000000000002e-05 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.487 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1185 3.f 3:f699bf02:::rbd_header.11365ca8c0d:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.590882+0000", 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8072707370000001, 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063611700000000002, 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1185 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.590882+0000", 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.590882+0000", 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.590885+0000", 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6440000000000002e-06 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.590901+0000", 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5991999999999999e-05 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.590941+0000", 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9685999999999999e-05 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.590956+0000", 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4956e-05 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.591052+0000", 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5563999999999995e-05 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:21.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.591061+0000", 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0860000000000008e-06 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.594864+0000", 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038037129999999998 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.594997+0000", 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132843 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.595594+0000", 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00059714499999999999 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.596768+0000", 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001174018 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.597167+0000", 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039863600000000001 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.597210+0000", 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3476999999999997e-05 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.597243+0000", 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2410000000000003e-05 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1325 3.1b e31/23 3:de203a46:::rbd_object_map.11365ca8c0d:head v 31'77, mlcod=31'72)", 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.729990+0000", 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.668162181, 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048861440000000003, 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.729990+0000", 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.729990+0000", 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.729991+0000", 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5799999999999998e-07 2024-09-16T14:07:21.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.729998+0000", 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.81e-06 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730014+0000", 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6532999999999999e-05 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730021+0000", 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5220000000000004e-06 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730080+0000", 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8940000000000002e-05 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730106+0000", 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6126999999999999e-05 2024-09-16T14:07:21.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.734831+0000", 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047252229999999997 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.734876+0000", 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5130999999999999e-05 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1589 3.3 3:c21d5247:::rbd_data.1136494ba971.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.891432+0000", 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.506720177, 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045877909999999999, 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1589 2024-09-16T14:07:21.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.891432+0000", 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.891432+0000", 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.891437+0000", 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2200000000000003e-06 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.891737+0000", 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030049100000000001 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.891769+0000", 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2283999999999998e-05 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.891782+0000", 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.255e-05 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.891860+0000", 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8589000000000003e-05 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.893691+0000", 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018305660000000001 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.894020+0000", 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032859399999999998 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.895820+0000", 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001800423 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.895921+0000", 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010074 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.895976+0000", 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5058e-05 2024-09-16T14:07:21.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.896020+0000", 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4276000000000001e-05 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:3821 3.3 3:c0e631c4:::rbd_header.1136494ba971:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:20.335127+0000", 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.063025533, 2024-09-16T14:07:21.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088664859999999998, 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3821 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.335127+0000", 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.335127+0000", 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.335129+0000", 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.677e-06 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.335137+0000", 2024-09-16T14:07:21.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3000000000000002e-06 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.335158+0000", 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1290000000000001e-05 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.335168+0000", 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4210000000000001e-06 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.335241+0000", 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3141999999999999e-05 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.335250+0000", 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7040000000000006e-06 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.339349+0000", 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040984589999999996 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.339487+0000", 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013816999999999999 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.339786+0000", 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029849400000000001 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.341346+0000", 2024-09-16T14:07:21.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001560503 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.343902+0000", 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025557399999999999 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.343958+0000", 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5886e-05 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.343993+0000", 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.57e-05 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:3887 3.3 3:c0e631c4:::rbd_header.1136494ba971:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:20.458880+0000", 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.93927222600000004, 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003929152, 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3887 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.458880+0000", 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.458880+0000", 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.458882+0000", 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0190000000000001e-06 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.458892+0000", 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6069999999999997e-06 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.458917+0000", 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5252000000000001e-05 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.458926+0000", 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1160000000000005e-06 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.459006+0000", 2024-09-16T14:07:21.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0013999999999997e-05 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.459135+0000", 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012918200000000001 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.459403+0000", 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000267125 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.460270+0000", 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086747300000000001 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.462709+0000", 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024389310000000001 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.462766+0000", 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6737000000000002e-05 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.462809+0000", 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3696000000000003e-05 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:3919 3.3 3:c0e631c4:::rbd_header.1136494ba971:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:21.340721+0000", 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.057431061999999998, 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.009021243, 2024-09-16T14:07:21.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3919 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.340721+0000", 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.340721+0000", 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.340724+0000", 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9000000000000002e-06 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.340737+0000", 2024-09-16T14:07:21.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2665e-05 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.340779+0000", 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2369e-05 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.340792+0000", 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2492e-05 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.340883+0000", 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1003999999999998e-05 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.340891+0000", 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2509999999999994e-06 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.345859+0000", 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049683119999999999 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.345949+0000", 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9417000000000001e-05 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.346390+0000", 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000441376 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.349596+0000", 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032054969999999999 2024-09-16T14:07:21.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.349622+0000", 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6466e-05 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.349697+0000", 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4418000000000005e-05 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:21.349743+0000", 2024-09-16T14:07:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6075999999999997e-05 2024-09-16T14:07:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.526 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:21.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:21.528 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:21.522+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:21.629 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:21.622+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:21.642 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:21.642 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:21.642 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:21.642 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:21.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:21.730 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:21.726+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:21.747 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:21.747 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:21.747 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:21.748 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:21.748 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:21.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:21.833 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:21.826+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:21.863 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.010882301999999, 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15070547300000001, 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847226+0000", 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.204e-06 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847235+0000", 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847250+0000", 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5662e-05 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847261+0000", 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0236e-05 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847327+0000", 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.622e-05 2024-09-16T14:07:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847345+0000", 2024-09-16T14:07:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8434e-05 2024-09-16T14:07:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.993706+0000", 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14636041899999999 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994117+0000", 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041132499999999998 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994428+0000", 2024-09-16T14:07:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031060899999999999 2024-09-16T14:07:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.995845+0000", 2024-09-16T14:07:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014173359999999999 2024-09-16T14:07:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997768+0000", 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001922991 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997867+0000", 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9057999999999999e-05 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997929+0000", 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2433000000000003e-05 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.225136+0000", 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.632970174, 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010268787, 2024-09-16T14:07:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 42 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225136+0000", 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225136+0000", 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225137+0000", 2024-09-16T14:07:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4499999999999996e-07 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225141+0000", 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5600000000000004e-06 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225150+0000", 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2889999999999998e-06 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.225155+0000", 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9110000000000001e-06 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.228294+0000", 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003139102 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.228355+0000", 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0863999999999999e-05 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.228460+0000", 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105392 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.232739+0000", 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042787149999999998 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235331+0000", 2024-09-16T14:07:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002592759 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235373+0000", 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1325999999999997e-05 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235405+0000", 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2023999999999997e-05 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226025+0000", 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.632080962, 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010082832999999999, 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226025+0000", 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226025+0000", 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226027+0000", 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4309999999999999e-06 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226035+0000", 2024-09-16T14:07:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2230000000000007e-06 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226058+0000", 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2878999999999999e-05 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226069+0000", 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1073999999999999e-05 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.231642+0000", 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055735689999999996 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.231751+0000", 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108142 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.231902+0000", 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015110300000000001 2024-09-16T14:07:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235420+0000", 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035178520000000001 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235943+0000", 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052378799999999997 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236043+0000", 2024-09-16T14:07:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9788000000000001e-05 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236108+0000", 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4984e-05 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226091+0000", 2024-09-16T14:07:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.632015324000001, 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011816414000000001, 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 46 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226091+0000", 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226091+0000", 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226092+0000", 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.08e-06 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226102+0000", 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226124+0000", 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2282000000000001e-05 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226146+0000", 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2087999999999999e-05 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.233144+0000", 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0069974499999999997 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.233335+0000", 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019087600000000001 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.233599+0000", 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026448699999999999 2024-09-16T14:07:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.235852+0000", 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022531270000000002 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237788+0000", 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00193542 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237847+0000", 2024-09-16T14:07:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8962999999999998e-05 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237907+0000", 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0751999999999997e-05 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226215+0000", 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.631891160999999, 2024-09-16T14:07:21.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011817500999999999, 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 47 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226215+0000", 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226215+0000", 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226217+0000", 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5149999999999999e-06 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226403+0000", 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018652100000000001 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226429+0000", 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6363000000000002e-05 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226447+0000", 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7342000000000001e-05 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.234097+0000", 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007650034 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.234228+0000", 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000131576 2024-09-16T14:07:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.234432+0000", 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020386800000000001 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236598+0000", 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021659919999999998 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237965+0000", 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013668770000000001 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238005+0000", 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0108999999999998e-05 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238033+0000", 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7304000000000002e-05 2024-09-16T14:07:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.631630669, 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013812964, 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 48 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226477+0000", 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.879999999999999e-07 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226482+0000", 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0919999999999997e-06 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226537+0000", 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4916000000000001e-05 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226546+0000", 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9709999999999999e-06 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236148+0000", 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0096011849999999999 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236208+0000", 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9922e-05 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236308+0000", 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010068000000000001 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239183+0000", 2024-09-16T14:07:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002874779 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240208+0000", 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001025312 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240255+0000", 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6363999999999999e-05 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240288+0000", 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3754999999999998e-05 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.631551567999999, 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013937603, 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 49 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0299999999999998e-07 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226560+0000", 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5879999999999999e-06 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226568+0000", 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3799999999999994e-06 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226575+0000", 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236676+0000", 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010100448999999999 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236851+0000", 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017497899999999999 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237123+0000", 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027242299999999999 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240139+0000", 2024-09-16T14:07:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030155109999999998 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240368+0000", 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000229461 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240417+0000", 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9045999999999998e-05 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240492+0000", 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5177000000000002e-05 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.631524679999998, 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015154025999999999, 2024-09-16T14:07:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 50 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226710+0000", 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012769999999999999 2024-09-16T14:07:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226721+0000", 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226726+0000", 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6459999999999998e-06 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238128+0000", 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011401390000000001 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238249+0000", 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121284 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238407+0000", 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015738899999999999 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240509+0000", 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002102923 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241459+0000", 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094996999999999996 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241621+0000", 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000161502 2024-09-16T14:07:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241736+0000", 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011458799999999999 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.631371755, 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015305664, 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 51 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226735+0000", 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5700000000000001e-07 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226740+0000", 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6460000000000001e-06 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226748+0000", 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226753+0000", 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6229999999999996e-06 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239228+0000", 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012475119999999999 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239405+0000", 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177641 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239559+0000", 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015368599999999999 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241393+0000", 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018340260000000001 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241863+0000", 2024-09-16T14:07:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047025300000000001 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241934+0000", 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0229999999999997e-05 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242040+0000", 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106568 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.631345361000001, 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015737872, 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 52 2024-09-16T14:07:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226762+0000", 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.23e-07 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226766+0000", 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.668e-06 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226774+0000", 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8939999999999993e-06 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226779+0000", 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1170000000000003e-06 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240584+0000", 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013804344 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240709+0000", 2024-09-16T14:07:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012524300000000001 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241014+0000", 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030534399999999998 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242062+0000", 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010479899999999999 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242369+0000", 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030710399999999999 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242442+0000", 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2383000000000001e-05 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242499+0000", 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.770661+0000", 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.087445398, 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011089416, 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 68 2024-09-16T14:07:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770661+0000", 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770661+0000", 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770663+0000", 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3199999999999998e-06 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770673+0000", 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0216e-05 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770694+0000", 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0942999999999999e-05 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770707+0000", 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2343e-05 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.776828+0000", 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061213450000000003 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.776928+0000", 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100012 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.777100+0000", 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017197600000000001 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781463+0000", 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043629669999999997 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781606+0000", 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014329100000000001 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781705+0000", 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9123999999999995e-05 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781750+0000", 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4879000000000002e-05 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.770777+0000", 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.087329187999998, 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011286358999999999, 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 69 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770777+0000", 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770777+0000", 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770779+0000", 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.95e-06 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770915+0000", 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013586699999999999 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770927+0000", 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2670000000000001e-05 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770939+0000", 2024-09-16T14:07:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1487e-05 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.777750+0000", 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068105800000000001 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.777871+0000", 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012134900000000001 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.778043+0000", 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000171953 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781843+0000", 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038004369999999998 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781864+0000", 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0831000000000001e-05 2024-09-16T14:07:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.781963+0000", 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9173999999999996e-05 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782063+0000", 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000100061 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.770966+0000", 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.087140617999999, 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012891682999999999, 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 70 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770966+0000", 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770966+0000", 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770967+0000", 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3060000000000001e-06 2024-09-16T14:07:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770976+0000", 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3500000000000003e-06 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.770995+0000", 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8756000000000001e-05 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771007+0000", 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.187e-05 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.779351+0000", 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083437349999999997 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.779443+0000", 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2261000000000003e-05 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.779629+0000", 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018625 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782202+0000", 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025728439999999999 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783776+0000", 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015739700000000001 2024-09-16T14:07:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783826+0000", 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9602999999999999e-05 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783857+0000", 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1738e-05 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.087084823000001, 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013869546999999999, 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 71 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771022+0000", 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0699999999999999e-06 2024-09-16T14:07:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771031+0000", 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3899999999999993e-06 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771048+0000", 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6949e-05 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771062+0000", 2024-09-16T14:07:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4647000000000001e-05 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780217+0000", 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091543300000000005 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780323+0000", 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106223 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780485+0000", 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016238299999999999 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784807+0000", 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043215500000000004 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784821+0000", 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4268e-05 2024-09-16T14:07:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784861+0000", 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9462000000000003e-05 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784891+0000", 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0275000000000001e-05 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.087030377000001, 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015694993000000001, 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 72 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771077+0000", 2024-09-16T14:07:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0299999999999997e-07 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771209+0000", 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132347 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771224+0000", 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.454e-05 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771230+0000", 2024-09-16T14:07:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:07:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782272+0000", 2024-09-16T14:07:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011041455 2024-09-16T14:07:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782402+0000", 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013039099999999999 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782645+0000", 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024303500000000001 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786509+0000", 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003863698 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786628+0000", 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011894499999999999 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786711+0000", 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2961999999999996e-05 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786771+0000", 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0098e-05 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.086867647999998, 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016692631999999999, 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 73 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771240+0000", 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.569999999999999e-07 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771244+0000", 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771253+0000", 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8629999999999997e-06 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771262+0000", 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783924+0000", 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012662479000000001 2024-09-16T14:07:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784015+0000", 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0624999999999994e-05 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784129+0000", 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113563 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787598+0000", 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034694890000000001 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787707+0000", 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109341 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787840+0000", 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132421 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787931+0000", 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1291000000000004e-05 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.086837551999999, 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017320314999999999, 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771278+0000", 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3790000000000006e-06 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771286+0000", 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5199999999999997e-06 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771292+0000", 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2839999999999997e-06 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784933+0000", 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013641304 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785019+0000", 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6201999999999999e-05 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785164+0000", 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014457599999999999 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787609+0000", 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024456769999999998 2024-09-16T14:07:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788403+0000", 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079383000000000003 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788490+0000", 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7231000000000006e-05 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788589+0000", 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8608000000000002e-05 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.086807841999999, 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017489688, 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771299+0000", 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3e-07 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771421+0000", 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121544 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771433+0000", 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853e-05 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771443+0000", 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0229999999999998e-06 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785601+0000", 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014158145 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785677+0000", 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6711999999999993e-05 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785774+0000", 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6583999999999995e-05 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787633+0000", 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001859461 2024-09-16T14:07:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788688+0000", 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010546349999999999 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788747+0000", 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.923e-05 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788788+0000", 2024-09-16T14:07:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0771e-05 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.086651262, 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017898397999999999, 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 76 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771456+0000", 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7900000000000007e-07 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771466+0000", 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8800000000000003e-06 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771475+0000", 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1670000000000005e-06 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771480+0000", 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.49e-06 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786862+0000", 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015381807000000001 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786985+0000", 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012302099999999999 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787142+0000", 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015696500000000001 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788157+0000", 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001014923 2024-09-16T14:07:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789175+0000", 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010175200000000001 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789259+0000", 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4578000000000002e-05 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789353+0000", 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4067999999999995e-05 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:55.579053+0000", 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.279053049000002, 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010202396000000001, 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 94 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579053+0000", 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579053+0000", 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579054+0000", 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8299999999999995e-07 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579059+0000", 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0429999999999998e-06 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579080+0000", 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0740000000000001e-05 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.579086+0000", 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9630000000000003e-06 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.585180+0000", 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060944880000000003 2024-09-16T14:07:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.585248+0000", 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7447999999999995e-05 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.585377+0000", 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012897599999999999 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.586525+0000", 2024-09-16T14:07:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011482420000000001 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.589165+0000", 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026399010000000001 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.589202+0000", 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6711999999999997e-05 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:55.589256+0000", 2024-09-16T14:07:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3999999999999998e-05 2024-09-16T14:07:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:21.991 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:21.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:21.993 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:21.986+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:22.093 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:22.086+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:22.143 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:22.143 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:22.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:22.194 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:22.190+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:22.256 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:22.257 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:22.257 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:22.257 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:22.257 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:22.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:22.295 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:22.290+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:22.397 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:22.390+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:22.404 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone3 from 0xe380509 (overlap 0x132c18d) to 0x0 2024-09-16T14:07:22.440 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.079811456, 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056564060000000001, 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357785+0000", 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0779999999999998e-06 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357794+0000", 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357811+0000", 2024-09-16T14:07:22.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7300999999999999e-05 2024-09-16T14:07:22.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357825+0000", 2024-09-16T14:07:22.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3959e-05 2024-09-16T14:07:22.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357934+0000", 2024-09-16T14:07:22.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010899 2024-09-16T14:07:22.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:22.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357942+0000", 2024-09-16T14:07:22.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.182e-06 2024-09-16T14:07:22.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.360955+0000", 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003013258 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361149+0000", 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019401600000000001 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361448+0000", 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029863300000000003 2024-09-16T14:07:22.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.362788+0000", 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013406480000000001 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363338+0000", 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054941500000000004 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363400+0000", 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2440999999999994e-05 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363440+0000", 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9734999999999998e-05 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:694 3.e 3:76483abd:::rbd_data.1136e0d3367d.000000000000016c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:22.451 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 6.6631799659999995, 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055514199999999996, 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 694 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:22.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774417+0000", 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0509999999999999e-06 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774424+0000", 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4229999999999999e-06 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774443+0000", 2024-09-16T14:07:22.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8664999999999998e-05 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774457+0000", 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3974e-05 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774531+0000", 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4208999999999994e-05 2024-09-16T14:07:22.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774612+0000", 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0438000000000005e-05 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.775728+0000", 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001116414 2024-09-16T14:07:22.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.777045+0000", 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013172139999999999 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779908+0000", 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002863071 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779939+0000", 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0840000000000003e-05 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779966+0000", 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7121e-05 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:830 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.635674+0000", 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.801920923, 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046520279999999999, 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.460 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 830 2024-09-16T14:07:22.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.460 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.635674+0000", 2024-09-16T14:07:22.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.635674+0000", 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.635677+0000", 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:07:22.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.635692+0000", 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5009e-05 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.635732+0000", 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0043000000000003e-05 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.635749+0000", 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6433e-05 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.635861+0000", 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111981 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.635871+0000", 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0876000000000001e-05 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.639875+0000", 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040031889999999999 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.640005+0000", 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013078699999999999 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.640326+0000", 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032056000000000001 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:882 3.e 3:728ab515:::rbd_data.1136e0d3367d.0000000000000008:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.712323+0000", 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7252716259999996, 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043022770000000002, 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 882 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712323+0000", 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712323+0000", 2024-09-16T14:07:22.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712327+0000", 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6720000000000002e-06 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712362+0000", 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5067000000000002e-05 2024-09-16T14:07:22.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712407+0000", 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4842000000000002e-05 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712430+0000", 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3348000000000001e-05 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712587+0000", 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015716800000000001 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712951+0000", 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00036350999999999999 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716625+0000", 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036746700000000001 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:887 3.16 3:6ac3ceae:::rbd_data.1136e0d3367d.000000000000000d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.712500+0000", 2024-09-16T14:07:22.471 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7250950070000002, 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037877620000000001, 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 887 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712500+0000", 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712500+0000", 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712502+0000", 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.717e-06 2024-09-16T14:07:22.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712521+0000", 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9579999999999999e-05 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712543+0000", 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2342000000000001e-05 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712556+0000", 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2206e-05 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712737+0000", 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018166499999999999 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.713069+0000", 2024-09-16T14:07:22.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033203300000000002 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716288+0000", 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003218219 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:889 3.5 3:a0d58f5c:::rbd_data.1136e0d3367d.000000000000000f:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.712589+0000", 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7250060539999996, 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038010240000000001, 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 889 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.477 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712589+0000", 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712589+0000", 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712590+0000", 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031e-06 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712596+0000", 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6279999999999999e-06 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712665+0000", 2024-09-16T14:07:22.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9066999999999994e-05 2024-09-16T14:07:22.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712673+0000", 2024-09-16T14:07:22.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.357e-06 2024-09-16T14:07:22.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712774+0000", 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010165799999999999 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.712956+0000", 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018175099999999999 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.713436+0000", 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047932600000000003 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716211+0000", 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027756299999999999 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716248+0000", 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6949000000000001e-05 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716321+0000", 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2657999999999995e-05 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716390+0000", 2024-09-16T14:07:22.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8968999999999996e-05 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:888 3.14 e30/23 3:2a25f6a6:::rbd_data.1136e0d3367d.000000000000000e:head v 30'4, mlcod=28'1)", 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.714397+0000", 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7231974049999996, 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004588562, 2024-09-16T14:07:22.484 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.714397+0000", 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.714397+0000", 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.714400+0000", 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999999e-06 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.714416+0000", 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6453e-05 2024-09-16T14:07:22.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.714443+0000", 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6811000000000001e-05 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.714457+0000", 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4243000000000001e-05 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.714574+0000", 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116828 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.714631+0000", 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6625000000000001e-05 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718935+0000", 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043042940000000002 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718986+0000", 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0908000000000001e-05 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:891 3.16 3:6f320eef:::rbd_data.1136e0d3367d.0000000000000010:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.716684+0000", 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7209108019999997, 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043043159999999999, 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 891 2024-09-16T14:07:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716684+0000", 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716684+0000", 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716686+0000", 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6950000000000001e-06 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716694+0000", 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2700000000000004e-06 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716724+0000", 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0446e-05 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716734+0000", 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6199999999999994e-06 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.716853+0000", 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011879099999999999 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717050+0000", 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000196916 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717272+0000", 2024-09-16T14:07:22.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000222313 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.720503+0000", 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032310160000000002 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.720727+0000", 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022402699999999999 2024-09-16T14:07:22.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.720844+0000", 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116667 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.720988+0000", 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014455499999999999 2024-09-16T14:07:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:895 3.e 3:728ab515:::rbd_data.1136e0d3367d.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.717082+0000", 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7205132299999999, 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040920699999999997, 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 895 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717082+0000", 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717082+0000", 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717084+0000", 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6460000000000002e-06 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717098+0000", 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4182000000000001e-05 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717187+0000", 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8138000000000002e-05 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717202+0000", 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5440000000000001e-05 2024-09-16T14:07:22.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717276+0000", 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3640999999999995e-05 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717417+0000", 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141085 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.717836+0000", 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041970100000000003 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.719333+0000", 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014962969999999999 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.721013+0000", 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001680321 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.721130+0000", 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116675 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.721174+0000", 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3943999999999998e-05 2024-09-16T14:07:22.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:896 3.16 3:6ac3ceae:::rbd_data.1136e0d3367d.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.718563+0000", 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7190314170000001, 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049910299999999996, 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.504 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 896 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718563+0000", 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718563+0000", 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718566+0000", 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3120000000000001e-06 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718589+0000", 2024-09-16T14:07:22.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3788e-05 2024-09-16T14:07:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718677+0000", 2024-09-16T14:07:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7681000000000003e-05 2024-09-16T14:07:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718692+0000", 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4501999999999999e-05 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718879+0000", 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018730999999999999 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718984+0000", 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105432 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.719238+0000", 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025403200000000001 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.721549+0000", 2024-09-16T14:07:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023106400000000001 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.723359+0000", 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001809677 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.723463+0000", 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010472 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.723554+0000", 2024-09-16T14:07:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0935999999999998e-05 2024-09-16T14:07:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:894 3.1 e30/23 3:834411d3:::rbd_data.1136e0d3367d.0000000000000011:head v 30'3, mlcod=0'0)", 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.718737+0000", 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.7188581100000002, 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046746729999999998, 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718737+0000", 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718737+0000", 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718738+0000", 2024-09-16T14:07:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2699999999999998e-07 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718763+0000", 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5765000000000001e-05 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718778+0000", 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4969e-05 2024-09-16T14:07:22.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718786+0000", 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1430000000000001e-06 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718873+0000", 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7188000000000006e-05 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718904+0000", 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1285000000000003e-05 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.723364+0000", 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044602729999999998 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.723411+0000", 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7123000000000003e-05 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1028 3.5 3:a45e2964:::rbd_data.11365ca8c0d.000000000000003b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.2814508839999998, 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051968600000000002, 2024-09-16T14:07:22.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1028 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156146+0000", 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5129999999999999e-06 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156159+0000", 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.276e-05 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156189+0000", 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0209000000000002e-05 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156210+0000", 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0647000000000001e-05 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156339+0000", 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012944000000000001 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156648+0000", 2024-09-16T14:07:22.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030876299999999998 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.161341+0000", 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046925279999999996 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1105 3.1 e30/23 3:86ae877e:::rbd_data.11365ca8c0d.000000000000001a:head v 30'6, mlcod=30'4)", 2024-09-16T14:07:22.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.257857+0000", 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.1797377749999995, 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003832266, 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.257857+0000", 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.257857+0000", 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.257858+0000", 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1400000000000001e-06 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.257985+0000", 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012686800000000001 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258000+0000", 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5218000000000001e-05 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258009+0000", 2024-09-16T14:07:22.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1810000000000007e-06 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258060+0000", 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0151999999999997e-05 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258078+0000", 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.889e-05 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.261637+0000", 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003558355 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.261689+0000", 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2462e-05 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1106 3.11 e30/23 3:8da88c9c:::rbd_data.11365ca8c0d.0000000000000014:head v 30'5, mlcod=28'1)", 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.1786736260000001, 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054785859999999997, 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258922+0000", 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9500000000000001e-07 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258929+0000", 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4560000000000003e-06 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258944+0000", 2024-09-16T14:07:22.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4548e-05 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258950+0000", 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9959999999999999e-06 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.259023+0000", 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2805000000000005e-05 2024-09-16T14:07:22.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.259056+0000", 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3395000000000001e-05 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.264352+0000", 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052958019999999996 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.264400+0000", 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7788999999999999e-05 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1246 3.5 3:a7daa15d:::rbd_id.image_client.0-clone3:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.664391+0000", 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.773204174, 2024-09-16T14:07:22.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037068689999999998, 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1246 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.664391+0000", 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.664391+0000", 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.664392+0000", 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8190000000000001e-06 2024-09-16T14:07:22.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.664401+0000", 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4980000000000006e-06 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.664425+0000", 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4348000000000001e-05 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.664434+0000", 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7919999999999998e-06 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.664536+0000", 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101549 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.664802+0000", 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026673000000000002 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.665245+0000", 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00044247500000000001 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.666720+0000", 2024-09-16T14:07:22.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014747269999999999 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.667990+0000", 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001270298 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.668053+0000", 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3309e-05 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.668098+0000", 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4323999999999998e-05 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1327 3.1e e31/23 3:7f5fe03f:::rbd_data.11365ca8c0d.000000000000000a:head v 31'3, mlcod=28'1)", 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.724188+0000", 2024-09-16T14:07:22.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7134069910000003, 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003548926, 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.724188+0000", 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.724188+0000", 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.724188+0000", 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9700000000000003e-07 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.724193+0000", 2024-09-16T14:07:22.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6569999999999996e-06 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.724205+0000", 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2330000000000001e-05 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.724211+0000", 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.724258+0000", 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6990999999999998e-05 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.724273+0000", 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4722000000000001e-05 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.727690+0000", 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034176340000000001 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.727737+0000", 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6448e-05 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1334 3.16 3:684be18d:::rbd_data.11365ca8c0d.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,truncate 1048576] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.730686+0000", 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7069086559999995, 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042072780000000001, 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1334 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730686+0000", 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730686+0000", 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730688+0000", 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3859999999999999e-06 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730693+0000", 2024-09-16T14:07:22.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6489999999999996e-06 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730708+0000", 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4612e-05 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730715+0000", 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3980000000000002e-06 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730782+0000", 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6667000000000004e-05 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.730983+0000", 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020084200000000001 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.731264+0000", 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028161499999999998 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.732494+0000", 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001229801 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.734781+0000", 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022864000000000001 2024-09-16T14:07:22.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.734841+0000", 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0602e-05 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.734893+0000", 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2305999999999997e-05 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1667 3.5 3:a3b2a978:::rbd_data.1136494ba971.00000000000001f9:head [set-alloc-hint object_size 262144 write_size 262144,write 246535~7417] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:18.010281+0000", 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4273135569999997, 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035524350000000001, 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1667 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010281+0000", 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010281+0000", 2024-09-16T14:07:22.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010283+0000", 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010296+0000", 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3049e-05 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010313+0000", 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7510999999999999e-05 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010321+0000", 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9240000000000007e-06 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010436+0000", 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011545 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010763+0000", 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032655800000000001 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.011084+0000", 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032086499999999998 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.012787+0000", 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001703558 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.013742+0000", 2024-09-16T14:07:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095459699999999995 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.013794+0000", 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.215e-05 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.013834+0000", 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9483000000000001e-05 2024-09-16T14:07:22.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1668 3.5 3:a4605f64:::rbd_data.1136494ba971.00000000000001fa:head [set-alloc-hint object_size 262144 write_size 262144,write 245760~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:18.010598+0000", 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4269964460000004, 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050155920000000001, 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1668 2024-09-16T14:07:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010598+0000", 2024-09-16T14:07:22.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010598+0000", 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010599+0000", 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.06e-06 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010614+0000", 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4258e-05 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010639+0000", 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5627999999999999e-05 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010649+0000", 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7540000000000001e-06 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.011990+0000", 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013405559999999999 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.012132+0000", 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014202 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.012290+0000", 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158782 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.015180+0000", 2024-09-16T14:07:22.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00288989 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.015534+0000", 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035365100000000002 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.015579+0000", 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4684000000000001e-05 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.015614+0000", 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5308999999999997e-05 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:3850 3.16 3:6fa6863f:::rbd_data.1136494ba971.000000000000021b:head [set-alloc-hint object_size 262144 write_size 262144,write 58251~7285] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:22.567 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.0569635750000002, 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052342250000000003, 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3850 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380633+0000", 2024-09-16T14:07:22.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8920000000000001e-06 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380647+0000", 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3455000000000001e-05 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380670+0000", 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3215999999999999e-05 2024-09-16T14:07:22.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380678+0000", 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2779999999999993e-06 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380734+0000", 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6255999999999998e-05 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380799+0000", 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4204999999999999e-05 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380900+0000", 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101672 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385645+0000", 2024-09-16T14:07:22.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047443859999999997 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385755+0000", 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110315 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385815+0000", 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0035999999999999e-05 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:22.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385865+0000", 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0513999999999998e-05 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:22.574 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:22.576 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:22.577 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:22.570+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:22.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:22.614 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-16T14:07:22.653 INFO:teuthology.orchestra.run.smithi079.stdout:599 read 0xdabec65 thru 0xdacc53c (0xd8d8 bytes) 2024-09-16T14:07:22.677 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:22.670+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:22.678 INFO:teuthology.orchestra.run.smithi079.stdout:600 trunc from 0xe380509 to 0x38def83 2024-09-16T14:07:22.684 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:22.684 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:22.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:22.781 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:22.774+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:22.798 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:22.836 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:22.837 INFO:teuthology.orchestra.run.smithi079.stdout:603 punch from 0xf4d6ff to 0xf551ee, (0x7aef bytes) 2024-09-16T14:07:22.837 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:22.837 INFO:teuthology.orchestra.run.smithi079.stdout:606 read 0x303acc9 thru 0x303f98f (0x4cc7 bytes) 2024-09-16T14:07:22.837 INFO:teuthology.orchestra.run.smithi079.stdout:607 read 0xf8adcf thru 0xf94e22 (0xa054 bytes) 2024-09-16T14:07:22.839 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:22.846 INFO:teuthology.orchestra.run.smithi079.stdout:609 trunc from 0x38def83 to 0x91b70f4 2024-09-16T14:07:22.856 INFO:teuthology.orchestra.run.smithi079.stdout:610 read 0x463ffb7 thru 0x4642e82 (0x2ecc bytes) 2024-09-16T14:07:22.862 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:22.863 INFO:teuthology.orchestra.run.smithi079.stdout:611 trunc from 0x91b70f4 to 0xe1d9fd0 2024-09-16T14:07:22.873 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:22.873 INFO:teuthology.orchestra.run.smithi079.stdout:613 write 0xac0abee thru 0xac184fe (0xd911 bytes) 2024-09-16T14:07:22.880 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:22.874+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:22.880 INFO:teuthology.orchestra.run.smithi079.stdout:614 punch from 0x67b80de to 0x67c6930, (0xe852 bytes) 2024-09-16T14:07:22.880 INFO:teuthology.orchestra.run.smithi079.stdout:615 write 0x78118c thru 0x78920a (0x807f bytes) 2024-09-16T14:07:22.892 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:22.900 INFO:teuthology.orchestra.run.smithi079.stdout:616 trunc from 0xe1d9fd0 to 0x2684173 2024-09-16T14:07:22.902 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:22.902 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:22.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:22.916 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:22.923 INFO:teuthology.orchestra.run.smithi079.stdout:618 read 0x1ac5910 thru 0x1acff1f (0xa610 bytes) 2024-09-16T14:07:22.924 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:22.924 INFO:teuthology.orchestra.run.smithi079.stdout:620 read 0x1c4a737 thru 0x1c4ca84 (0x234e bytes) 2024-09-16T14:07:22.925 INFO:teuthology.orchestra.run.smithi079.stdout:623 read 0x542ab3 thru 0x544262 (0x17b0 bytes) 2024-09-16T14:07:22.936 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:22.951 INFO:teuthology.orchestra.run.smithi079.stdout:624 write 0xd108b25 thru 0xd10ce2c (0x4308 bytes) 2024-09-16T14:07:22.958 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:22.970 INFO:teuthology.orchestra.run.smithi079.stdout:625 write 0xe931008 thru 0xe932a94 (0x1a8d bytes) 2024-09-16T14:07:22.976 INFO:teuthology.orchestra.run.smithi079.stdout:626 writesame 0xce79b75 thru 0xce8912e data_size 0x2(0xf5ba bytes) 2024-09-16T14:07:22.980 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:22.974+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:22.981 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:22.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:22.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:22.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:22.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:22.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:22.985 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:22.985 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:22.985 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:23.011 INFO:teuthology.orchestra.run.smithi079.stdout:628 trunc from 0xe932a95 to 0x4a609bd 2024-09-16T14:07:23.015 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.015 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:23.026 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:23.038 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:23.038 INFO:teuthology.orchestra.run.smithi079.stdout:630 write 0x18ed4a0 thru 0x18fa832 (0xd393 bytes) 2024-09-16T14:07:23.041 INFO:teuthology.orchestra.run.smithi079.stdout:631 write 0x1f37b33 thru 0x1f3f566 (0x7a34 bytes) 2024-09-16T14:07:23.064 INFO:teuthology.orchestra.run.smithi079.stdout:632 write 0x81ef8ca thru 0x81fe582 (0xecb9 bytes) 2024-09-16T14:07:23.068 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:23.076 INFO:teuthology.orchestra.run.smithi079.stdout:635 trunc from 0x81fe583 to 0x497dfa3 2024-09-16T14:07:23.081 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:23.074+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:23.099 INFO:teuthology.orchestra.run.smithi079.stdout:637 read 0x38a5495 thru 0x38adbad (0x8719 bytes) 2024-09-16T14:07:23.100 INFO:teuthology.orchestra.run.smithi079.stdout:638 read 0x4542b71 thru 0x454abce (0x805e bytes) 2024-09-16T14:07:23.117 INFO:teuthology.orchestra.run.smithi079.stdout:639 trunc from 0x497dfa3 to 0xdec39ab 2024-09-16T14:07:23.129 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.129 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:23.135 INFO:teuthology.orchestra.run.smithi079.stdout:640 trunc from 0xdec39ab to 0xa2fdf76 2024-09-16T14:07:23.139 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:23.152 INFO:teuthology.orchestra.run.smithi079.stdout:641 trunc from 0xa2fdf76 to 0x8054354 2024-09-16T14:07:23.164 INFO:teuthology.orchestra.run.smithi079.stdout:642 write 0x7f5796e thru 0x7f60001 (0x8694 bytes) 2024-09-16T14:07:23.182 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:23.174+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:23.187 INFO:teuthology.orchestra.run.smithi079.stdout:643 write 0x997d6aa thru 0x9989c3c (0xc593 bytes) 2024-09-16T14:07:23.197 INFO:teuthology.orchestra.run.smithi079.stdout:644 read 0x85717e5 thru 0x857294f (0x116b bytes) 2024-09-16T14:07:23.197 INFO:teuthology.orchestra.run.smithi079.stdout:645 punch from 0x3fe3f71 to 0x3febe0b, (0x7e9a bytes) 2024-09-16T14:07:23.207 INFO:teuthology.orchestra.run.smithi079.stdout:646 trunc from 0x9989c3d to 0xdbaed2b 2024-09-16T14:07:23.220 INFO:teuthology.orchestra.run.smithi079.stdout:647 write 0x6d0c3f2 thru 0x6d0f335 (0x2f44 bytes) 2024-09-16T14:07:23.227 INFO:teuthology.orchestra.run.smithi079.stdout:648 write 0x92d02e4 thru 0x92d14dc (0x11f9 bytes) 2024-09-16T14:07:23.230 INFO:teuthology.orchestra.run.smithi079.stdout:649 writesame 0x935c3b1 thru 0x9366951 data_size 0x1(0xa5a1 bytes) 2024-09-16T14:07:23.240 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.240 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:23.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:23.259 INFO:teuthology.orchestra.run.smithi079.stdout:651 punch from 0x21ab918 to 0x21abbbe, (0x2a6 bytes) 2024-09-16T14:07:23.259 INFO:teuthology.orchestra.run.smithi079.stdout:652 read 0x57e6e5a thru 0x57f0434 (0x95db bytes) 2024-09-16T14:07:23.259 INFO:teuthology.orchestra.run.smithi079.stdout:654 read 0x53e5a71 thru 0x53ea317 (0x48a7 bytes) 2024-09-16T14:07:23.260 INFO:teuthology.orchestra.run.smithi079.stdout:655 clone 5 order 21 su 1048576 sc 8 2024-09-16T14:07:23.283 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:23.278+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:23.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:07:23.384 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:23.378+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:23.400 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.400 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:23.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:23.485 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:23.478+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:23.508 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.508 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:23.517 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:23.586 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:23.582+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:23.613 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.613 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:23.622 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:23.687 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:23.678+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:23.716 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.717 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:23.726 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:23.781 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone4 from 0xdbaed2b (overlap 0x2684173) to 0x13420b9 2024-09-16T14:07:23.787 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:23.778+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:23.882 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.882 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:23.888 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:23.882+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:23.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:23.967 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-16T14:07:23.984 INFO:teuthology.orchestra.run.smithi079.stdout:656 writesame 0x134a712 thru 0x1354d1e data_size 0x1(0xa60d bytes) 2024-09-16T14:07:23.988 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:23.982+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:23.992 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:23.992 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.003 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:24.023 INFO:teuthology.orchestra.run.smithi079.stdout:658 punch from 0x6300942 to 0x630762c, (0x6cea bytes) 2024-09-16T14:07:24.023 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.023 INFO:teuthology.orchestra.run.smithi079.stdout:661 read 0x1c5eeb7 thru 0x1c60eba (0x2004 bytes) 2024-09-16T14:07:24.024 INFO:teuthology.orchestra.run.smithi079.stdout:662 read 0x86f1d90 thru 0x86f7349 (0x55ba bytes) 2024-09-16T14:07:24.025 INFO:teuthology.orchestra.run.smithi079.stdout:664 write 0x97d2589 thru 0x97e1d9f (0xf817 bytes) 2024-09-16T14:07:24.036 INFO:teuthology.orchestra.run.smithi079.stdout:665 read 0x420dda thru 0x42962e (0x8855 bytes) 2024-09-16T14:07:24.037 INFO:teuthology.orchestra.run.smithi079.stdout:669 punch from 0x8e0132c to 0x8e07f2a, (0x6bfe bytes) 2024-09-16T14:07:24.037 INFO:teuthology.orchestra.run.smithi079.stdout:670 read 0xb36c741 thru 0xb36fd97 (0x3657 bytes) 2024-09-16T14:07:24.058 INFO:teuthology.orchestra.run.smithi079.stdout:671 trunc from 0xdbaed2b to 0x3884c28 2024-09-16T14:07:24.090 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:24.082+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:24.096 INFO:teuthology.orchestra.run.smithi079.stdout:672 read 0xcf97b4 thru 0xd00206 (0x6a53 bytes) 2024-09-16T14:07:24.098 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.106 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:24.106 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.117 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:24.131 INFO:teuthology.orchestra.run.smithi079.stdout:674 writesame 0xc9b45d1 thru 0xc9c15b6 data_size 0x59(0xcfe6 bytes) 2024-09-16T14:07:24.131 INFO:teuthology.orchestra.run.smithi079.stdout:675 trunc from 0xc9c15b7 to 0xcb136ea 2024-09-16T14:07:24.140 INFO:teuthology.orchestra.run.smithi079.stdout:677 writesame 0x186caf7 thru 0x186cf4b data_size 0x1(0x455 bytes) 2024-09-16T14:07:24.152 INFO:teuthology.orchestra.run.smithi079.stdout:678 read 0x2856c32 thru 0x285c8b5 (0x5c84 bytes) 2024-09-16T14:07:24.153 INFO:teuthology.orchestra.run.smithi079.stdout:680 read 0xc59b8a thru 0xc69ac8 (0xff3f bytes) 2024-09-16T14:07:24.155 INFO:teuthology.orchestra.run.smithi079.stdout:681 write 0xb850da8 thru 0xb85bda3 (0xaffc bytes) 2024-09-16T14:07:24.162 INFO:teuthology.orchestra.run.smithi079.stdout:683 read 0x2397297 thru 0x23a5338 (0xe0a2 bytes) 2024-09-16T14:07:24.164 INFO:teuthology.orchestra.run.smithi079.stdout:685 read 0x2e90139 thru 0x2e97355 (0x721d bytes) 2024-09-16T14:07:24.175 INFO:teuthology.orchestra.run.smithi079.stdout:686 trunc from 0xcb136ea to 0x95178c3 2024-09-16T14:07:24.191 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:24.186+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:24.193 INFO:teuthology.orchestra.run.smithi079.stdout:688 read 0x3d35bb thru 0x3e2611 (0xf057 bytes) 2024-09-16T14:07:24.195 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.196 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.196 INFO:teuthology.orchestra.run.smithi079.stdout:693 writesame 0x8029966 thru 0x8032713 data_size 0x3(0x8dae bytes) 2024-09-16T14:07:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:24.215 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.224 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:24.225 INFO:teuthology.orchestra.run.smithi079.stdout:696 writesame 0xb561e08 thru 0xb5640e8 data_size 0x1(0x22e1 bytes) 2024-09-16T14:07:24.225 INFO:teuthology.orchestra.run.smithi079.stdout:698 punch from 0x821cfbd to 0x822448d, (0x74d0 bytes) 2024-09-16T14:07:24.247 INFO:teuthology.orchestra.run.smithi079.stdout:699 trunc from 0xb5640e9 to 0x4e153 2024-09-16T14:07:24.280 INFO:teuthology.orchestra.run.smithi079.stdout:700 read 0x1f359 thru 0x2aa6d (0xb715 bytes) 2024-09-16T14:07:24.281 INFO:teuthology.orchestra.run.smithi079.stdout:701 read 0x13649 thru 0x18e74 (0x582c bytes) 2024-09-16T14:07:24.292 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:24.286+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:24.315 INFO:teuthology.orchestra.run.smithi079.stdout:702 write 0xc74a52a thru 0xc74eed1 (0x49a8 bytes) 2024-09-16T14:07:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:24.318 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.320 INFO:teuthology.orchestra.run.smithi079.stdout:704 read 0x7a0801f thru 0x7a11b75 (0x9b57 bytes) 2024-09-16T14:07:24.320 INFO:teuthology.orchestra.run.smithi079.stdout:705 writesame 0x87c19e3 thru 0x87c3c43 data_size 0x1(0x2261 bytes) 2024-09-16T14:07:24.327 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:24.330 INFO:teuthology.orchestra.run.smithi079.stdout:706 punch from 0x8af52a2 to 0x8b04853, (0xf5b1 bytes) 2024-09-16T14:07:24.342 INFO:teuthology.orchestra.run.smithi079.stdout:707 write 0xcc62f77 thru 0xcc6edf5 (0xbe7f bytes) 2024-09-16T14:07:24.349 INFO:teuthology.orchestra.run.smithi079.stdout:710 punch from 0x84d28f3 to 0x84db398, (0x8aa5 bytes) 2024-09-16T14:07:24.349 INFO:teuthology.orchestra.run.smithi079.stdout:711 punch from 0x35fc2ff to 0x360a3aa, (0xe0ab bytes) 2024-09-16T14:07:24.349 INFO:teuthology.orchestra.run.smithi079.stdout:712 punch from 0x56367b9 to 0x563aa54, (0x429b bytes) 2024-09-16T14:07:24.349 INFO:teuthology.orchestra.run.smithi079.stdout:713 writesame 0x4496abd thru 0x449d6f9 data_size 0xb(0x6c3d bytes) 2024-09-16T14:07:24.357 INFO:teuthology.orchestra.run.smithi079.stdout:714 write 0x39acfa6 thru 0x39bace3 (0xdd3e bytes) 2024-09-16T14:07:24.364 INFO:teuthology.orchestra.run.smithi079.stdout:715 write 0x36bbb8a thru 0x36c82b6 (0xc72d bytes) 2024-09-16T14:07:24.390 INFO:teuthology.orchestra.run.smithi079.stdout:716 trunc from 0xcc6edf6 to 0x455b5b9 2024-09-16T14:07:24.393 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:24.386+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:24.413 INFO:teuthology.orchestra.run.smithi079.stdout:717 write 0x9b3f05 thru 0x9b5d70 (0x1e6c bytes) 2024-09-16T14:07:24.419 INFO:teuthology.orchestra.run.smithi079.stdout:718 write 0x23b511b thru 0x23b83b0 (0x3296 bytes) 2024-09-16T14:07:24.425 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.425 INFO:teuthology.orchestra.run.smithi079.stdout:721 read 0x3ed9e8c thru 0x3ee2c7e (0x8df3 bytes) 2024-09-16T14:07:24.426 INFO:teuthology.orchestra.run.smithi079.stdout:722 write 0x43c2623 thru 0x43cc852 (0xa230 bytes) 2024-09-16T14:07:24.457 INFO:teuthology.orchestra.run.smithi079.stdout:723 write 0xa038b39 thru 0xa0451b8 (0xc680 bytes) 2024-09-16T14:07:24.463 INFO:teuthology.orchestra.run.smithi079.stdout:725 read 0x43cbd1e thru 0x43d6331 (0xa614 bytes) 2024-09-16T14:07:24.465 INFO:teuthology.orchestra.run.smithi079.stdout:726 writesame 0x62e969c thru 0x62f77ec data_size 0x11(0xe151 bytes) 2024-09-16T14:07:24.475 INFO:teuthology.orchestra.run.smithi079.stdout:727 read 0x184cee4 thru 0x18595b2 (0xc6cf bytes) 2024-09-16T14:07:24.475 INFO:teuthology.orchestra.run.smithi079.stdout:728 read 0x21f198f thru 0x21fcb7f (0xb1f1 bytes) 2024-09-16T14:07:24.475 INFO:teuthology.orchestra.run.smithi079.stdout:729 read 0x423c1dc thru 0x423f7e4 (0x3609 bytes) 2024-09-16T14:07:24.477 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:24.477 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:24.494 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:24.486+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:24.498 INFO:teuthology.orchestra.run.smithi079.stdout:731 write 0xe5b0988 thru 0xe5bd6d5 (0xcd4e bytes) 2024-09-16T14:07:24.504 INFO:teuthology.orchestra.run.smithi079.stdout:734 punch from 0xaa3484f to 0xaa36f3b, (0x26ec bytes) 2024-09-16T14:07:24.504 INFO:teuthology.orchestra.run.smithi079.stdout:736 writesame 0x2456688 thru 0x2461fe1 data_size 0x32(0xb95a bytes) 2024-09-16T14:07:24.511 INFO:teuthology.orchestra.run.smithi079.stdout:738 write 0xb742125 thru 0xb74e8c0 (0xc79c bytes) 2024-09-16T14:07:24.516 INFO:teuthology.orchestra.run.smithi079.stdout:740 writesame 0x7812341 thru 0x7820a8a data_size 0x3e(0xe74a bytes) 2024-09-16T14:07:24.523 INFO:teuthology.orchestra.run.smithi079.stdout:741 read 0xc17a2e2 thru 0xc1854ca (0xb1e9 bytes) 2024-09-16T14:07:24.523 INFO:teuthology.orchestra.run.smithi079.stdout:742 write 0x17c1e9e thru 0x17c96fe (0x7861 bytes) 2024-09-16T14:07:24.530 INFO:teuthology.orchestra.run.smithi079.stdout:743 write 0x6cc32f7 thru 0x6ccafb6 (0x7cc0 bytes) 2024-09-16T14:07:24.536 INFO:teuthology.orchestra.run.smithi079.stdout:744 write 0x4ff3cc3 thru 0x4fffccc (0xc00a bytes) 2024-09-16T14:07:24.543 INFO:teuthology.orchestra.run.smithi079.stdout:746 punch from 0x7ae75b4 to 0x7af1126, (0x9b72 bytes) 2024-09-16T14:07:24.543 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.543 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.543 INFO:teuthology.orchestra.run.smithi079.stdout:751 punch from 0xa996fe7 to 0xa9a2650, (0xb669 bytes) 2024-09-16T14:07:24.543 INFO:teuthology.orchestra.run.smithi079.stdout:752 read 0x59f690a thru 0x59fa709 (0x3e00 bytes) 2024-09-16T14:07:24.543 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.543 INFO:teuthology.orchestra.run.smithi079.stdout:755 writesame 0x6f65f05 thru 0x6f6fe2c data_size 0x2c(0x9f28 bytes) 2024-09-16T14:07:24.548 INFO:teuthology.orchestra.run.smithi079.stdout:758 read 0x4afb054 thru 0x4b01154 (0x6101 bytes) 2024-09-16T14:07:24.562 INFO:teuthology.orchestra.run.smithi079.stdout:759 write 0xed77923 thru 0xed7c3e4 (0x4ac2 bytes) 2024-09-16T14:07:24.565 INFO:teuthology.orchestra.run.smithi079.stdout:761 punch from 0xc11d0d0 to 0xc11fbb4, (0x2ae4 bytes) 2024-09-16T14:07:24.565 INFO:teuthology.orchestra.run.smithi079.stdout:762 punch from 0x40c0e6c to 0x40c4b62, (0x3cf6 bytes) 2024-09-16T14:07:24.565 INFO:teuthology.orchestra.run.smithi079.stdout:763 write 0xbd15014 thru 0xbd24788 (0xf775 bytes) 2024-09-16T14:07:24.572 INFO:teuthology.orchestra.run.smithi079.stdout:765 write 0x9a38541 thru 0x9a43671 (0xb131 bytes) 2024-09-16T14:07:24.577 INFO:teuthology.orchestra.run.smithi079.stdout:766 write 0x631a5d7 thru 0x6321a31 (0x745b bytes) 2024-09-16T14:07:24.581 INFO:teuthology.orchestra.run.smithi079.stdout:769 punch from 0x4e728cb to 0x4e77df9, (0x552e bytes) 2024-09-16T14:07:24.581 INFO:teuthology.orchestra.run.smithi079.stdout:770 write 0x5ff493c thru 0x600348f (0xeb54 bytes) 2024-09-16T14:07:24.588 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.588 INFO:teuthology.orchestra.run.smithi079.stdout:772 write 0x873859c thru 0x87440ec (0xbb51 bytes) 2024-09-16T14:07:24.592 INFO:teuthology.orchestra.run.smithi079.stdout:773 punch from 0x228b568 to 0x2298dc2, (0xd85a bytes) 2024-09-16T14:07:24.595 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:24.586+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:24.602 INFO:teuthology.orchestra.run.smithi079.stdout:774 trunc from 0xed7c3e5 to 0x9a0acb1 2024-09-16T14:07:24.605 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:24.605 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.615 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:24.630 INFO:teuthology.orchestra.run.smithi079.stdout:775 read 0x90ad7c8 thru 0x90b6d86 (0x95bf bytes) 2024-09-16T14:07:24.630 INFO:teuthology.orchestra.run.smithi079.stdout:776 punch from 0x5b34630 to 0x5b3b9b2, (0x7382 bytes) 2024-09-16T14:07:24.630 INFO:teuthology.orchestra.run.smithi079.stdout:778 read 0x29b12bb thru 0x29bd0b9 (0xbdff bytes) 2024-09-16T14:07:24.630 INFO:teuthology.orchestra.run.smithi079.stdout:779 write 0x7fbdc0e thru 0x7fc4e67 (0x725a bytes) 2024-09-16T14:07:24.631 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:07:24.659 INFO:teuthology.orchestra.run.smithi079.stdout:780 write 0xb355f2a thru 0xb365115 (0xf1ec bytes) 2024-09-16T14:07:24.664 INFO:teuthology.orchestra.run.smithi079.stdout:782 punch from 0x1830a6f to 0x1830cd7, (0x268 bytes) 2024-09-16T14:07:24.664 INFO:teuthology.orchestra.run.smithi079.stdout:783 read 0x14db682 thru 0x14dea0b (0x338a bytes) 2024-09-16T14:07:24.664 INFO:teuthology.orchestra.run.smithi079.stdout:784 read 0x200efdb thru 0x201a8df (0xb905 bytes) 2024-09-16T14:07:24.664 INFO:teuthology.orchestra.run.smithi079.stdout:785 writesame 0x8af4770 thru 0x8afa384 data_size 0x1(0x5c15 bytes) 2024-09-16T14:07:24.686 INFO:teuthology.orchestra.run.smithi079.stdout:786 trunc from 0xb365116 to 0x987421b 2024-09-16T14:07:24.698 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:24.690+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:24.706 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.706 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:24.706 INFO:teuthology.orchestra.run.smithi079.stdout:789 clone 6 order 20 su 131072 sc 7 2024-09-16T14:07:24.724 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:24.725 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:24.798 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:24.794+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:24.876 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:24.876 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:24.901 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:24.894+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:24.984 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:24.985 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:24.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:25.002 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:24.994+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:25.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:25.089 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:25.089 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:25.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:25.105 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:25.102+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:25.134 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:25.155 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:25.176 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:25.196 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:25.196 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:25.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:25.196 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:25.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 31.346431726999999, 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15070547300000001, 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.197 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847226+0000", 2024-09-16T14:07:25.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.204e-06 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847235+0000", 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847250+0000", 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5662e-05 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847261+0000", 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0236e-05 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847327+0000", 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.622e-05 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847345+0000", 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8434e-05 2024-09-16T14:07:25.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.993706+0000", 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14636041899999999 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994117+0000", 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041132499999999998 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994428+0000", 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031060899999999999 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.995845+0000", 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014173359999999999 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997768+0000", 2024-09-16T14:07:25.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001922991 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997867+0000", 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9057999999999999e-05 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997929+0000", 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2433000000000003e-05 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.967180094, 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013812964, 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.204 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 48 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226477+0000", 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.879999999999999e-07 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226482+0000", 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0919999999999997e-06 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226537+0000", 2024-09-16T14:07:25.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4916000000000001e-05 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226546+0000", 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9709999999999999e-06 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236148+0000", 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0096011849999999999 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236208+0000", 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9922e-05 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236308+0000", 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010068000000000001 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239183+0000", 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002874779 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240208+0000", 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001025312 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240255+0000", 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6363999999999999e-05 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240288+0000", 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3754999999999998e-05 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.967100992999999, 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013937603, 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 49 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0299999999999998e-07 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226560+0000", 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5879999999999999e-06 2024-09-16T14:07:25.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226568+0000", 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3799999999999994e-06 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226575+0000", 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236676+0000", 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010100448999999999 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236851+0000", 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017497899999999999 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237123+0000", 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027242299999999999 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240139+0000", 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030155109999999998 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240368+0000", 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000229461 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240417+0000", 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9045999999999998e-05 2024-09-16T14:07:25.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240492+0000", 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5177000000000002e-05 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.967074105000002, 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015154025999999999, 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 50 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:25.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226710+0000", 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012769999999999999 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226721+0000", 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226726+0000", 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6459999999999998e-06 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238128+0000", 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011401390000000001 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238249+0000", 2024-09-16T14:07:25.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121284 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238407+0000", 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015738899999999999 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240509+0000", 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002102923 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241459+0000", 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094996999999999996 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241621+0000", 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000161502 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241736+0000", 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011458799999999999 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.222 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.96692118, 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015305664, 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.223 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 51 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226735+0000", 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5700000000000001e-07 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226740+0000", 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6460000000000001e-06 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226748+0000", 2024-09-16T14:07:25.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226753+0000", 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6229999999999996e-06 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239228+0000", 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012475119999999999 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239405+0000", 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177641 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239559+0000", 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015368599999999999 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241393+0000", 2024-09-16T14:07:25.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018340260000000001 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241863+0000", 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047025300000000001 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241934+0000", 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0229999999999997e-05 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242040+0000", 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106568 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.966894786000001, 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015737872, 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.229 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 52 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226762+0000", 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.23e-07 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226766+0000", 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.668e-06 2024-09-16T14:07:25.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226774+0000", 2024-09-16T14:07:25.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8939999999999993e-06 2024-09-16T14:07:25.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226779+0000", 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1170000000000003e-06 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240584+0000", 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013804344 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240709+0000", 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012524300000000001 2024-09-16T14:07:25.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241014+0000", 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030534399999999998 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242062+0000", 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010479899999999999 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242369+0000", 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030710399999999999 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242442+0000", 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2383000000000001e-05 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242499+0000", 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:07:25.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.422634248000001, 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013869546999999999, 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 71 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771022+0000", 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0699999999999999e-06 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771031+0000", 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3899999999999993e-06 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771048+0000", 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6949e-05 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771062+0000", 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4647000000000001e-05 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780217+0000", 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091543300000000005 2024-09-16T14:07:25.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780323+0000", 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106223 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780485+0000", 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016238299999999999 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784807+0000", 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043215500000000004 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784821+0000", 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4268e-05 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784861+0000", 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9462000000000003e-05 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784891+0000", 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0275000000000001e-05 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.422579802000001, 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015694993000000001, 2024-09-16T14:07:25.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 72 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771077+0000", 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0299999999999997e-07 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771209+0000", 2024-09-16T14:07:25.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132347 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771224+0000", 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.454e-05 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771230+0000", 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782272+0000", 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011041455 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782402+0000", 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013039099999999999 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782645+0000", 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024303500000000001 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786509+0000", 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003863698 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786628+0000", 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011894499999999999 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786711+0000", 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2961999999999996e-05 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786771+0000", 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0098e-05 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.248 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.422417072999998, 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016692631999999999, 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 73 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.249 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771240+0000", 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.569999999999999e-07 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771244+0000", 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771253+0000", 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8629999999999997e-06 2024-09-16T14:07:25.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771262+0000", 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783924+0000", 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012662479000000001 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784015+0000", 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0624999999999994e-05 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784129+0000", 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113563 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787598+0000", 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034694890000000001 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787707+0000", 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109341 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787840+0000", 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132421 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787931+0000", 2024-09-16T14:07:25.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1291000000000004e-05 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.422386976999999, 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017320314999999999, 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771278+0000", 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3790000000000006e-06 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771286+0000", 2024-09-16T14:07:25.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5199999999999997e-06 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771292+0000", 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2839999999999997e-06 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784933+0000", 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013641304 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785019+0000", 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6201999999999999e-05 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785164+0000", 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014457599999999999 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787609+0000", 2024-09-16T14:07:25.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024456769999999998 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788403+0000", 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079383000000000003 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788490+0000", 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7231000000000006e-05 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788589+0000", 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8608000000000002e-05 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.422357266999999, 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017489688, 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771299+0000", 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3e-07 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771421+0000", 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121544 2024-09-16T14:07:25.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771433+0000", 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853e-05 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771443+0000", 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0229999999999998e-06 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785601+0000", 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014158145 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785677+0000", 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6711999999999993e-05 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785774+0000", 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6583999999999995e-05 2024-09-16T14:07:25.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787633+0000", 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001859461 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788688+0000", 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010546349999999999 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788747+0000", 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.923e-05 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788788+0000", 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0771e-05 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:25.267 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 30.422200687, 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017898397999999999, 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 76 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771456+0000", 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7900000000000007e-07 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771466+0000", 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8800000000000003e-06 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771475+0000", 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1670000000000005e-06 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771480+0000", 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.49e-06 2024-09-16T14:07:25.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786862+0000", 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015381807000000001 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786985+0000", 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012302099999999999 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787142+0000", 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015696500000000001 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788157+0000", 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001014923 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789175+0000", 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010175200000000001 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789259+0000", 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4578000000000002e-05 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789353+0000", 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4067999999999995e-05 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.273 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5396 3.1a 3:5f43d5fe:::rbd_data.1136494ba971.0000000000000020:7 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723923+0000", 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.469732391, 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0131158, 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5396 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723923+0000", 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723923+0000", 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723924+0000", 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723925+0000", 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8500000000000001e-06 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723931+0000", 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6300000000000003e-06 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723935+0000", 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5779999999999999e-06 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.736875+0000", 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012940161 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.736953+0000", 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8169000000000003e-05 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737039+0000", 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6020999999999995e-05 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.278 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5397 3.1a 3:5f43d5fe:::rbd_data.1136494ba971.0000000000000020:7 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.4697172470000002, 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013283031000000001, 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5397 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723939+0000", 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.03e-07 2024-09-16T14:07:25.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723941+0000", 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7910000000000001e-06 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723947+0000", 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.798e-06 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723951+0000", 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.546e-06 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737068+0000", 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013117199 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737147+0000", 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8473999999999994e-05 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737221+0000", 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4820000000000005e-05 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5470 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 73728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.469594039, 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013300935, 2024-09-16T14:07:25.283 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5470 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724064+0000", 2024-09-16T14:07:25.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724069+0000", 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2050000000000001e-06 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724074+0000", 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7049999999999998e-06 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737243+0000", 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013169273 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737309+0000", 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6087000000000006e-05 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737363+0000", 2024-09-16T14:07:25.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3341999999999999e-05 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5471 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.469573848, 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013448857, 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5471 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3300000000000002e-07 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724085+0000", 2024-09-16T14:07:25.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3460000000000001e-06 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724090+0000", 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7189999999999998e-06 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724094+0000", 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-06 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737386+0000", 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013292438 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737461+0000", 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4554999999999993e-05 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737531+0000", 2024-09-16T14:07:25.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9726000000000003e-05 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5472 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.4695562529999999, 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013595701, 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5472 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:25.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724100+0000", 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8500000000000002e-07 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724108+0000", 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9759999999999995e-06 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724113+0000", 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6180000000000003e-06 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724117+0000", 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0550000000000001e-06 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737557+0000", 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013439184 2024-09-16T14:07:25.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737633+0000", 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6682999999999997e-05 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737695+0000", 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1799999999999998e-05 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5523 3.10 3:0ab9df7f:::rbd_data.1136494ba971.0000000000000065:7 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724309+0000", 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.4693467089999999, 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013271132999999999, 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5523 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724309+0000", 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724309+0000", 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724309+0000", 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:07:25.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724311+0000", 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8560000000000001e-06 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724317+0000", 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1510000000000001e-06 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724321+0000", 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4369999999999999e-06 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737415+0000", 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013093746 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737500+0000", 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5829999999999996e-05 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737580+0000", 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9684999999999993e-05 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5757 3.1a 3:5c45bf46:::rbd_object_map.11369af4c40c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.46047832, 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01336064, 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5757 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733178+0000", 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8100000000000003e-07 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733181+0000", 2024-09-16T14:07:25.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4570000000000001e-06 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733188+0000", 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3589999999999996e-06 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733192+0000", 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0629999999999999e-06 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739596+0000", 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064040140000000004 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739669+0000", 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3559999999999994e-05 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.740040+0000", 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00037030500000000003 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.745915+0000", 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058753859999999998 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746458+0000", 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054278499999999995 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746503+0000", 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4811999999999997e-05 2024-09-16T14:07:25.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746538+0000", 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5417999999999999e-05 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:7540 3.10 3:0c0a7e4b:::rbd_data.11369af4c40c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.386934933, 2024-09-16T14:07:25.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016499916, 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7540 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:25.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806722+0000", 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.78e-07 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807847+0000", 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001125381 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807858+0000", 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1219999999999999e-05 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807863+0000", 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1569999999999999e-06 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808087+0000", 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022387000000000001 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808219+0000", 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013168199999999999 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.809044+0000", 2024-09-16T14:07:25.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082524200000000001 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.814222+0000", 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005177414 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823075+0000", 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088537470000000004 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823150+0000", 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4571000000000001e-05 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823221+0000", 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0654000000000005e-05 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.314 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:25.315 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:25.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:25.317 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:25.317 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:25.317 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:25.317 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:25.318 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:25.320 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:25.314+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:25.421 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:25.414+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:25.469 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:25.469 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:25.469 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:25.469 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:25.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:25.522 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:25.518+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:25.585 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:25.585 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:25.585 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:25.585 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:25.586 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:25.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:25.623 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:25.618+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:25.707 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:25.707 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:25.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:25.707 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:25.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.708 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:25.708 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:25.708 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.344532071, 2024-09-16T14:07:25.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056564060000000001, 2024-09-16T14:07:25.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.709 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12 2024-09-16T14:07:25.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.709 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:25.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:25.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357785+0000", 2024-09-16T14:07:25.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0779999999999998e-06 2024-09-16T14:07:25.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357794+0000", 2024-09-16T14:07:25.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:07:25.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357811+0000", 2024-09-16T14:07:25.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7300999999999999e-05 2024-09-16T14:07:25.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357825+0000", 2024-09-16T14:07:25.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3959e-05 2024-09-16T14:07:25.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357934+0000", 2024-09-16T14:07:25.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010899 2024-09-16T14:07:25.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357942+0000", 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.182e-06 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.360955+0000", 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003013258 2024-09-16T14:07:25.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361149+0000", 2024-09-16T14:07:25.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019401600000000001 2024-09-16T14:07:25.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:25.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361448+0000", 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029863300000000003 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.362788+0000", 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013406480000000001 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363338+0000", 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054941500000000004 2024-09-16T14:07:25.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363400+0000", 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2440999999999994e-05 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363440+0000", 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9734999999999998e-05 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:694 3.e 3:76483abd:::rbd_data.1136e0d3367d.000000000000016c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.9279005809999994, 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055514199999999996, 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 694 2024-09-16T14:07:25.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774417+0000", 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0509999999999999e-06 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774424+0000", 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4229999999999999e-06 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774443+0000", 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8664999999999998e-05 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774457+0000", 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3974e-05 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774531+0000", 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4208999999999994e-05 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774612+0000", 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0438000000000005e-05 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.775728+0000", 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001116414 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.777045+0000", 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013172139999999999 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779908+0000", 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002863071 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779939+0000", 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0840000000000003e-05 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779966+0000", 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7121e-05 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:896 3.16 3:6ac3ceae:::rbd_data.1136e0d3367d.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:16.718563+0000", 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9837520319999999, 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049910299999999996, 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 896 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718563+0000", 2024-09-16T14:07:25.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718563+0000", 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718566+0000", 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3120000000000001e-06 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718589+0000", 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3788e-05 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718677+0000", 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7681000000000003e-05 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718692+0000", 2024-09-16T14:07:25.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4501999999999999e-05 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718879+0000", 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018730999999999999 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.718984+0000", 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105432 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.719238+0000", 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025403200000000001 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.721549+0000", 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023106400000000001 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.723359+0000", 2024-09-16T14:07:25.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001809677 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.723463+0000", 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010472 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:16.723554+0000", 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0935999999999998e-05 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1028 3.5 3:a45e2964:::rbd_data.11365ca8c0d.000000000000003b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.5461714989999997, 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051968600000000002, 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1028 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156146+0000", 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5129999999999999e-06 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156159+0000", 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.276e-05 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156189+0000", 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0209000000000002e-05 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156210+0000", 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0647000000000001e-05 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156339+0000", 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012944000000000001 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156648+0000", 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030876299999999998 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.161341+0000", 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046925279999999996 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1106 3.11 e30/23 3:8da88c9c:::rbd_data.11365ca8c0d.0000000000000014:head v 30'5, mlcod=28'1)", 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.443394241, 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054785859999999997, 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:25.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258922+0000", 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9500000000000001e-07 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258929+0000", 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4560000000000003e-06 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258944+0000", 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4548e-05 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258950+0000", 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9959999999999999e-06 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.259023+0000", 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2805000000000005e-05 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.259056+0000", 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3395000000000001e-05 2024-09-16T14:07:25.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.264352+0000", 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052958019999999996 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.264400+0000", 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7788999999999999e-05 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1668 3.5 3:a4605f64:::rbd_data.1136494ba971.00000000000001fa:head [set-alloc-hint object_size 262144 write_size 262144,write 245760~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:18.010598+0000", 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.6917170610000003, 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050155920000000001, 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.745 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1668 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010598+0000", 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010598+0000", 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010599+0000", 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.06e-06 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010614+0000", 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4258e-05 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010639+0000", 2024-09-16T14:07:25.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5627999999999999e-05 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.010649+0000", 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7540000000000001e-06 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.011990+0000", 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013405559999999999 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.012132+0000", 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014202 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.012290+0000", 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158782 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.015180+0000", 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00288989 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.015534+0000", 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035365100000000002 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.015579+0000", 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4684000000000001e-05 2024-09-16T14:07:25.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:18.015614+0000", 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5308999999999997e-05 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:3850 3.16 3:6fa6863f:::rbd_data.1136494ba971.000000000000021b:head [set-alloc-hint object_size 262144 write_size 262144,write 58251~7285] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:25.751 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.32168419, 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052342250000000003, 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3850 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380633+0000", 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8920000000000001e-06 2024-09-16T14:07:25.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380647+0000", 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3455000000000001e-05 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380670+0000", 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3215999999999999e-05 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380678+0000", 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2779999999999993e-06 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380734+0000", 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6255999999999998e-05 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380799+0000", 2024-09-16T14:07:25.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4204999999999999e-05 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380900+0000", 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101672 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385645+0000", 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047443859999999997 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385755+0000", 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110315 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385815+0000", 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0035999999999999e-05 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385865+0000", 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0513999999999998e-05 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:4089 3.1e e35/23 3:7fd910ea:::rbd_data.1136494ba971.0000000000000011:head v 35'26, mlcod=35'22)", 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.421505+0000", 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.2808108369999998, 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051727919999999998, 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.421505+0000", 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.421505+0000", 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.421506+0000", 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0160000000000001e-06 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.421515+0000", 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6360000000000006e-06 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.421538+0000", 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2963e-05 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.421546+0000", 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0560000000000005e-06 2024-09-16T14:07:25.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.421639+0000", 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2929000000000004e-05 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.421667+0000", 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8090999999999999e-05 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426644+0000", 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049765599999999997 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426677+0000", 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3541000000000003e-05 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:4098 3.1e e35/23 3:783d22fa:::rbd_data.1136494ba971.0000000000000017:head v 35'28, mlcod=35'22)", 2024-09-16T14:07:25.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.2780417800000001, 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00715775, 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6499999999999998e-07 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424283+0000", 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6840000000000002e-06 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424295+0000", 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2074e-05 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424302+0000", 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.489e-06 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424345+0000", 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2846999999999998e-05 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424364+0000", 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9034000000000001e-05 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431403+0000", 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070398259999999999 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431431+0000", 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8031e-05 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4102 3.16 3:6ee147bb:::rbd_data.1136494ba971.000000000000001b:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.2772779339999998, 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063615549999999996, 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4102 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425038+0000", 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0400000000000005e-07 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425042+0000", 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9090000000000002e-06 2024-09-16T14:07:25.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425056+0000", 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3308e-05 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425060+0000", 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7820000000000001e-06 2024-09-16T14:07:25.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425105+0000", 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4833000000000003e-05 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425174+0000", 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9125e-05 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425313+0000", 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013855900000000001 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426552+0000", 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00123953 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431313+0000", 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004760401 2024-09-16T14:07:25.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431361+0000", 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8535999999999997e-05 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431399+0000", 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7667999999999999e-05 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4105 3.5 3:a4636173:::rbd_data.1136494ba971.000000000000001e:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.275582521, 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078170070000000008, 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4105 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426734+0000", 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.088e-06 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426739+0000", 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8550000000000001e-06 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426749+0000", 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0336999999999999e-05 2024-09-16T14:07:25.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426755+0000", 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426798+0000", 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2579e-05 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426862+0000", 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3493999999999999e-05 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426974+0000", 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112271 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.429506+0000", 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025319439999999999 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434430+0000", 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049244240000000002 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434501+0000", 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1023000000000001e-05 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434550+0000", 2024-09-16T14:07:25.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8678999999999999e-05 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4127 3.5 3:a4205384:::rbd_data.1136494ba971.0000000000000032:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.265597047, 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054291670000000004, 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4127 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436719+0000", 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9999999999999995e-07 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436724+0000", 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436739+0000", 2024-09-16T14:07:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.482e-05 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436744+0000", 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.775e-06 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436791+0000", 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7420000000000003e-05 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436886+0000", 2024-09-16T14:07:25.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4942e-05 2024-09-16T14:07:25.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:25.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437094+0000", 2024-09-16T14:07:25.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020822399999999999 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.439088+0000", 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019939340000000002 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.441989+0000", 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029012360000000002 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.442047+0000", 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.442148+0000", 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010031600000000001 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:4128 3.1 e35/23 3:80589f64:::rbd_data.1136494ba971.0000000000000033:head v 35'39, mlcod=35'36)", 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.437471+0000", 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.2648442009999998, 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050360329999999997, 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437471+0000", 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437471+0000", 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437472+0000", 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437503+0000", 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1167999999999997e-05 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437516+0000", 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3053e-05 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437522+0000", 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437568+0000", 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5090999999999999e-05 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437586+0000", 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8666000000000001e-05 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.442469+0000", 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048826750000000004 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.442507+0000", 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8343000000000002e-05 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4161 3.2 3:467b8b19:::rbd_data.1136494ba971.000000000000005a:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.791 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.2480852950000001, 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054131559999999997, 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4161 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454232+0000", 2024-09-16T14:07:25.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.756e-06 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454239+0000", 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9110000000000004e-06 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454259+0000", 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9749999999999999e-05 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454266+0000", 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8569999999999995e-06 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454334+0000", 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7104e-05 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454500+0000", 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016668100000000001 2024-09-16T14:07:25.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454703+0000", 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020301299999999999 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.457949+0000", 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032455309999999999 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:25.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459554+0000", 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016048 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459599+0000", 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5089999999999997e-05 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459643+0000", 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4663000000000003e-05 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5126 3.16 3:6ee147bb:::rbd_data.1136494ba971.000000000000001b:7 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.717677+0000", 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.9846383240000001, 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047770679999999998, 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5126 2024-09-16T14:07:25.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717677+0000", 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717677+0000", 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717678+0000", 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0999999999999999e-07 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717687+0000", 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0440000000000001e-06 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717694+0000", 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3769999999999998e-06 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717698+0000", 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2209999999999999e-06 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.719299+0000", 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00160085 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.722356+0000", 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030572080000000001 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.722454+0000", 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7857999999999997e-05 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5236 3.16 3:68fd61e8:::rbd_data.1136494ba971.0000000000000049:7 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.718767+0000", 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.9835480350000001, 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047912709999999997, 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5236 2024-09-16T14:07:25.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718767+0000", 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718767+0000", 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718768+0000", 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718775+0000", 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9770000000000003e-06 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718781+0000", 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.127e-06 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718784+0000", 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4429999999999999e-06 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723428+0000", 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046435369999999997 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723501+0000", 2024-09-16T14:07:25.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3331000000000007e-05 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723559+0000", 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7445000000000003e-05 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5293 3.e 3:70ce87e4:::rbd_data.1136494ba971.0000000000000071:7 [read 32768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.718875+0000", 2024-09-16T14:07:25.807 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.983440892, 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048932799999999999, 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5293 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718875+0000", 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718875+0000", 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718875+0000", 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718877+0000", 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0379999999999998e-06 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718907+0000", 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0434e-05 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718912+0000", 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.921e-06 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723583+0000", 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046708770000000004 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723697+0000", 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113987 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723768+0000", 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0600999999999997e-05 2024-09-16T14:07:25.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5294 3.e 3:70ce87e4:::rbd_data.1136494ba971.0000000000000071:7 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.718918+0000", 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.9833978669999999, 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050034229999999999, 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5294 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718918+0000", 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718918+0000", 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718918+0000", 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0299999999999999e-07 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718921+0000", 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5600000000000001e-06 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718927+0000", 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5640000000000002e-06 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718931+0000", 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.63e-06 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723790+0000", 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004858724 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723861+0000", 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1811999999999996e-05 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723921+0000", 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9630000000000003e-05 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5303 3.e 3:73080399:::rbd_data.1136494ba971.0000000000000075:7 [read 32768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.718941+0000", 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.9833740020000001, 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051456189999999997, 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5303 2024-09-16T14:07:25.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718941+0000", 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718941+0000", 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718942+0000", 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4499999999999997e-07 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718944+0000", 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.119e-06 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718950+0000", 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1619999999999996e-06 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718955+0000", 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9099999999999996e-06 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723954+0000", 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049994089999999998 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724029+0000", 2024-09-16T14:07:25.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4203999999999996e-05 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724087+0000", 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8369999999999998e-05 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5304 3.e 3:73080399:::rbd_data.1136494ba971.0000000000000075:7 [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:25.821 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.9833557449999999, 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052879110000000002, 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5304 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:25.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:25.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1800000000000001e-07 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718962+0000", 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.03e-06 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718969+0000", 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2490000000000004e-06 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718983+0000", 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3145999999999999e-05 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724107+0000", 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051242450000000004 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724185+0000", 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8077999999999993e-05 2024-09-16T14:07:25.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724248+0000", 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2744999999999996e-05 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:25.826 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:25.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:25.828 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone5 from 0x987421b (overlap 0x4e153) to 0x36a86 2024-09-16T14:07:25.829 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:25.822+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:25.928 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:25.922+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:25.929 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:25.930 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:25.930 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:25.930 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:25.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:26.030 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:26.026+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:26.038 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-16T14:07:26.048 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:26.048 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:26.049 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:26.049 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:26.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:26.116 INFO:teuthology.orchestra.run.smithi079.stdout:790 clone 7 order 23 su 524288 sc 3 2024-09-16T14:07:26.131 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:26.122+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:26.160 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:26.160 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:26.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:26.160 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:26.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.160 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 25.677049931999999, 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.095465191000000005, 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479100+0000", 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9840000000000001e-06 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479107+0000", 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7920000000000004e-06 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479120+0000", 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3195e-05 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479131+0000", 2024-09-16T14:07:26.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1416999999999999e-05 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479210+0000", 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8223e-05 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479224+0000", 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.436e-05 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570355+0000", 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091131223999999997 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570668+0000", 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031290000000000002 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570990+0000", 2024-09-16T14:07:26.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032147600000000001 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574367+0000", 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033769680000000002 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574482+0000", 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115021 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574528+0000", 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6035999999999998e-05 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574560+0000", 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2595000000000002e-05 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5239 3.9 3:95baad33:::rbd_data.1136494ba971.000000000000004b:7 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.167 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.717355+0000", 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4387898969999999, 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082845720000000005, 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5239 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717355+0000", 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717355+0000", 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717356+0000", 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717358+0000", 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8300000000000001e-06 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717365+0000", 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.985e-06 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717368+0000", 2024-09-16T14:07:26.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-06 2024-09-16T14:07:26.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725446+0000", 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0080779709999999998 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725546+0000", 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9932000000000005e-05 2024-09-16T14:07:26.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725640+0000", 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3461999999999994e-05 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5240 3.9 3:95baad33:::rbd_data.1136494ba971.000000000000004b:7 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.717373+0000", 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4387716830000001, 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092396019999999995, 2024-09-16T14:07:26.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5240 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717373+0000", 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717373+0000", 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717374+0000", 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:07:26.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717376+0000", 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9130000000000001e-06 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717382+0000", 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7970000000000003e-06 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717386+0000", 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5470000000000001e-06 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.726478+0000", 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090922039999999996 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.726553+0000", 2024-09-16T14:07:26.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5049999999999995e-05 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.726613+0000", 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9701999999999999e-05 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5241 3.11 3:8d1496bc:::rbd_data.1136494ba971.000000000000004c:7 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.717390+0000", 2024-09-16T14:07:26.178 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4387554040000001, 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0084286929999999993, 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5241 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717390+0000", 2024-09-16T14:07:26.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717390+0000", 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717390+0000", 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2699999999999999e-07 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717393+0000", 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.435e-06 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717399+0000", 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4529999999999999e-06 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717403+0000", 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6270000000000002e-06 2024-09-16T14:07:26.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725667+0000", 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082639759999999993 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725742+0000", 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5288000000000002e-05 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725818+0000", 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6386999999999999e-05 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5242 3.11 3:8d1496bc:::rbd_data.1136494ba971.000000000000004c:7 [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.717406+0000", 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.438738898, 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010497918, 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5242 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717406+0000", 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717406+0000", 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717406+0000", 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4299999999999999e-07 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717408+0000", 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5820000000000001e-06 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717421+0000", 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3297e-05 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717427+0000", 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3990000000000003e-06 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725840+0000", 2024-09-16T14:07:26.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0084134840000000006 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.727821+0000", 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019805479999999999 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.727904+0000", 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3264999999999996e-05 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5243 3.11 3:8d1496bc:::rbd_data.1136494ba971.000000000000004c:7 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.717431+0000", 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.438714241, 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012225969, 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5243 2024-09-16T14:07:26.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717431+0000", 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717431+0000", 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717431+0000", 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8599999999999999e-07 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717433+0000", 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2290000000000002e-06 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717439+0000", 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9349999999999999e-06 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.717443+0000", 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7469999999999999e-06 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729516+0000", 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01207251 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729593+0000", 2024-09-16T14:07:26.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7768000000000005e-05 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729657+0000", 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3393999999999996e-05 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5481 3.9 3:952933dc:::rbd_data.1136494ba971.0000000000000044:7 [read 65536~5470] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724685+0000", 2024-09-16T14:07:26.192 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.431460183, 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0084872109999999997, 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5481 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724685+0000", 2024-09-16T14:07:26.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724685+0000", 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724685+0000", 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724687+0000", 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9e-06 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724695+0000", 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9759999999999995e-06 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724699+0000", 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4120000000000001e-06 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732334+0000", 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076355829999999996 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732415+0000", 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0532999999999996e-05 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733172+0000", 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075740699999999998 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5482 3.9 3:952933dc:::rbd_data.1136494ba971.0000000000000044:7 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724702+0000", 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.431442605, 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086517609999999991, 2024-09-16T14:07:26.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5482 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724702+0000", 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724702+0000", 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724703+0000", 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.08e-07 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724705+0000", 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8059999999999999e-06 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724710+0000", 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3739999999999997e-06 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724713+0000", 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0819999999999999e-06 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733196+0000", 2024-09-16T14:07:26.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0084832859999999996 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733277+0000", 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0747999999999993e-05 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733354+0000", 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7057000000000004e-05 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5483 3.9 3:952933dc:::rbd_data.1136494ba971.0000000000000044:7 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724717+0000", 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4314284310000001, 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088330830000000003, 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5483 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.202 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724717+0000", 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724717+0000", 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724717+0000", 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8599999999999999e-07 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724719+0000", 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.852e-06 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724727+0000", 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3450000000000006e-06 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724731+0000", 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6239999999999999e-06 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733384+0000", 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086529239999999993 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733461+0000", 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7521999999999994e-05 2024-09-16T14:07:26.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733550+0000", 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8430000000000005e-05 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5497 3.9 3:95baad33:::rbd_data.1136494ba971.000000000000004b:7 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724735+0000", 2024-09-16T14:07:26.206 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4314103330000001, 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094444549999999992, 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5497 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724735+0000", 2024-09-16T14:07:26.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724735+0000", 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724735+0000", 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9799999999999999e-07 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724737+0000", 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7600000000000001e-06 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724742+0000", 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4380000000000003e-06 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724746+0000", 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4300000000000002e-06 2024-09-16T14:07:26.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733576+0000", 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088306110000000004 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733653+0000", 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6922000000000007e-05 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734179+0000", 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052589599999999998 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5498 3.9 3:95baad33:::rbd_data.1136494ba971.000000000000004b:7 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724749+0000", 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4313955539999998, 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0096215099999999998, 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.211 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5498 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724749+0000", 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724749+0000", 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724750+0000", 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7800000000000002e-07 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724751+0000", 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6270000000000001e-06 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724757+0000", 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4219999999999999e-06 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724760+0000", 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4709999999999999e-06 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734207+0000", 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094469410000000004 2024-09-16T14:07:26.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734285+0000", 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7349999999999996e-05 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734371+0000", 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6321000000000002e-05 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5499 3.9 3:95baad33:::rbd_data.1136494ba971.000000000000004b:7 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724764+0000", 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.431380973, 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010425583, 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5499 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724764+0000", 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724764+0000", 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724764+0000", 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724766+0000", 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.623e-06 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724771+0000", 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3519999999999997e-06 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724776+0000", 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3239999999999996e-06 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734498+0000", 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097224330000000008 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.735121+0000", 2024-09-16T14:07:26.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00062250100000000002 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.735190+0000", 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8966000000000003e-05 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5500 3.11 3:8d1496bc:::rbd_data.1136494ba971.000000000000004c:7 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724780+0000", 2024-09-16T14:07:26.220 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4313654360000001, 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097788590000000009, 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5500 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724780+0000", 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724780+0000", 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724780+0000", 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7500000000000001e-07 2024-09-16T14:07:26.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724782+0000", 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6309999999999999e-06 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724787+0000", 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5570000000000003e-06 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724791+0000", 2024-09-16T14:07:26.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.439e-06 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734394+0000", 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0096038370000000005 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734473+0000", 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8354000000000002e-05 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734558+0000", 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5666000000000003e-05 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5501 3.11 3:8d1496bc:::rbd_data.1136494ba971.000000000000004c:7 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724794+0000", 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4313508270000002, 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0099401980000000008, 2024-09-16T14:07:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5501 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724794+0000", 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724794+0000", 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724795+0000", 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2399999999999999e-07 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724797+0000", 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4499999999999998e-06 2024-09-16T14:07:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724819+0000", 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1656999999999999e-05 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724823+0000", 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.437e-06 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734581+0000", 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097577410000000003 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734654+0000", 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2706000000000005e-05 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.734734+0000", 2024-09-16T14:07:26.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0783000000000002e-05 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5743 3.9 3:95baad33:::rbd_data.1136494ba971.000000000000004b:7 [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.732014+0000", 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4241311840000002, 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0080250949999999995, 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.230 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5743 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732014+0000", 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732014+0000", 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732014+0000", 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.46e-07 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732016+0000", 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1440000000000001e-06 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732022+0000", 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5790000000000003e-06 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732025+0000", 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.518e-06 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739844+0000", 2024-09-16T14:07:26.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078185930000000004 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739931+0000", 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6431e-05 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.740039+0000", 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108384 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5745 3.11 3:8d1496bc:::rbd_data.1136494ba971.000000000000004c:7 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.732044+0000", 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4241014299999999, 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0077749830000000001, 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5745 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.235 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732044+0000", 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732044+0000", 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732044+0000", 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732046+0000", 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.995e-06 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732051+0000", 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4060000000000004e-06 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.732055+0000", 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.138e-06 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739692+0000", 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076365840000000001 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739759+0000", 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7186000000000004e-05 2024-09-16T14:07:26.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739819+0000", 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9307e-05 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:6150 3.13 3:cce811f8:::rbd_data.1136494ba971.0000000000000028:7 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.752660+0000", 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.403484985, 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0084194850000000009, 2024-09-16T14:07:26.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 6150 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.752660+0000", 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.752660+0000", 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.752660+0000", 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.27e-07 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.752662+0000", 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0250000000000001e-06 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.752668+0000", 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3720000000000004e-06 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.752673+0000", 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3569999999999997e-06 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.753101+0000", 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00042741099999999999 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.753186+0000", 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5185999999999994e-05 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.761079+0000", 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078937069999999998 2024-09-16T14:07:26.243 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:7538 3.9 3:92a13b8a:::rbd_data.11369af4c40c.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.812012+0000", 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.344132713, 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014792043, 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.244 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7538 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812012+0000", 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812012+0000", 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812013+0000", 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7400000000000003e-07 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812723+0000", 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070998800000000005 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812743+0000", 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0154000000000001e-05 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812750+0000", 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0299999999999996e-06 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812809+0000", 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9243000000000002e-05 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.813089+0000", 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027919700000000002 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.813791+0000", 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070247500000000004 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.819855+0000", 2024-09-16T14:07:26.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060638649999999999 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.826689+0000", 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068336179999999996 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.826756+0000", 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7724000000000004e-05 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.826804+0000", 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8075000000000003e-05 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:7541 3.13 3:ca9168fb:::rbd_data.11369af4c40c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.812766+0000", 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.3433793189999998, 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013303534000000001, 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7541 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812766+0000", 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812766+0000", 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.812767+0000", 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.691e-06 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.813513+0000", 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074583400000000002 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.813526+0000", 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3057000000000001e-05 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.813534+0000", 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9389999999999997e-06 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.813595+0000", 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0832000000000003e-05 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.813728+0000", 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013267199999999999 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.814486+0000", 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075862399999999995 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.824302+0000", 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0098154039999999998 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.825981+0000", 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001679247 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.826037+0000", 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5645e-05 2024-09-16T14:07:26.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.826069+0000", 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2589000000000003e-05 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:7547 3.13 3:c85e5295:::rbd_data.11369af4c40c.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.815463+0000", 2024-09-16T14:07:26.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.340682363, 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011727339, 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7547 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.815463+0000", 2024-09-16T14:07:26.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.815463+0000", 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.815464+0000", 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.31e-06 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:26.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.815808+0000", 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000344147 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.815825+0000", 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7008999999999999e-05 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.815833+0000", 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1429999999999998e-06 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.818875+0000", 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030416039999999998 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.819046+0000", 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000170891 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.820094+0000", 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010480730000000001 2024-09-16T14:07:26.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.824753+0000", 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046587180000000001 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.827103+0000", 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023507929999999999 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:26.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.827149+0000", 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5874000000000002e-05 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.827190+0000", 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0776999999999999e-05 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:26.262 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:26.264 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:26.264 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:26.262+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:26.360 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:07:26.360 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:07:26.360 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-16T14:07:26.360 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T14:07:26.360 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T14:07:26.361 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-16T14:07:26.364 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:26.358+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:26.371 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:26.371 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:26.371 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:26.371 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:26.382 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:26.465 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:26.462+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:26.474 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:26.475 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T14:07:26.475 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:26.489 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:26.489 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:26.490 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:26.506 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:26.566 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:26.558+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:26.606 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:918 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.8694910270000005, 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015774501, 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 918 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730965+0000", 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730967+0000", 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730971+0000", 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5080000000000001e-06 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730973+0000", 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.091e-06 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.739992+0000", 2024-09-16T14:07:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090191430000000003 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740162+0000", 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016968899999999999 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740834+0000", 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067256100000000001 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.744120+0000", 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032859370000000001 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746643+0000", 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025232050000000002 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746700+0000", 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6251000000000001e-05 2024-09-16T14:07:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746739+0000", 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9201e-05 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1032 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.156596+0000", 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.4438592440000004, 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014023004, 2024-09-16T14:07:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1032 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156596+0000", 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156596+0000", 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156598+0000", 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.17e-06 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156610+0000", 2024-09-16T14:07:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1653e-05 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156644+0000", 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3550000000000002e-05 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156661+0000", 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7218999999999999e-05 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.164337+0000", 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0076762740000000003 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.164455+0000", 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011811000000000001 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.164911+0000", 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00045541000000000001 2024-09-16T14:07:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.166666+0000", 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001755714 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170545+0000", 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038791619999999998 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170588+0000", 2024-09-16T14:07:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2849000000000003e-05 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170619+0000", 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0892999999999997e-05 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1033 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.156905+0000", 2024-09-16T14:07:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.4435505939999995, 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013772747, 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1033 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156905+0000", 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156905+0000", 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156907+0000", 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.103e-06 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156918+0000", 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0849000000000001e-05 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.156987+0000", 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9738000000000002e-05 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.157006+0000", 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8647999999999998e-05 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.165391+0000", 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083852749999999993 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.165496+0000", 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010478 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.165949+0000", 2024-09-16T14:07:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00045310000000000001 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.167897+0000", 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001947278 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170627+0000", 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002730564 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170656+0000", 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8575999999999999e-05 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.170678+0000", 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1835999999999999e-05 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1140 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336815+0000", 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.263640079, 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013885798, 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1140 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336815+0000", 2024-09-16T14:07:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336815+0000", 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336816+0000", 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6000000000000003e-07 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336820+0000", 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5650000000000002e-06 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336828+0000", 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4470000000000006e-06 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336834+0000", 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0100000000000001e-06 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345608+0000", 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0087740609999999997 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345792+0000", 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000184272 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.346482+0000", 2024-09-16T14:07:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00068944299999999998 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349525+0000", 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030429879999999999 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350582+0000", 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010570340000000001 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350623+0000", 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1029000000000003e-05 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350701+0000", 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8288999999999995e-05 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1141 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2636162199999994, 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014042171000000001, 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1141 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336840+0000", 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2600000000000002e-07 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336844+0000", 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7419999999999999e-06 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336850+0000", 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336855+0000", 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4939999999999997e-06 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347139+0000", 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010284331000000001 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347252+0000", 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011294300000000001 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347714+0000", 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046169700000000001 2024-09-16T14:07:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350720+0000", 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030062359999999998 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350774+0000", 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3415999999999998e-05 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350837+0000", 2024-09-16T14:07:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3151999999999994e-05 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350881+0000", 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4591e-05 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1142 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2635959700000008, 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016069268000000001, 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1142 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336860+0000", 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336866+0000", 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5600000000000001e-06 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336872+0000", 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9410000000000001e-06 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336877+0000", 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.172e-06 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348737+0000", 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011860239999999999 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348829+0000", 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1657999999999996e-05 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349140+0000", 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031145499999999998 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352798+0000", 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036581550000000002 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352815+0000", 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6883000000000001e-05 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352876+0000", 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0871e-05 2024-09-16T14:07:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352929+0000", 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2781000000000002e-05 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1143 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2635745709999995, 2024-09-16T14:07:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018939362000000001, 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1143 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.44e-07 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336884+0000", 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:07:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336891+0000", 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336895+0000", 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0820000000000001e-06 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349584+0000", 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012689283000000001 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349708+0000", 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123793 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350151+0000", 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044259900000000001 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352993+0000", 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028426979999999998 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355720+0000", 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002726642 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355762+0000", 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2116000000000001e-05 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355820+0000", 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8118999999999997e-05 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2635564650000006, 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:07:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:07:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:07:26.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2635340680000002, 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.656 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:07:26.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:07:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2629273570000006, 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:07:26.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:26.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:07:26.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:07:26.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2628530300000005, 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:07:26.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1150 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2555031099999994, 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017570243999999999, 2024-09-16T14:07:26.675 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1150 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344954+0000", 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4959999999999999e-06 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344964+0000", 2024-09-16T14:07:26.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8719999999999997e-06 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344980+0000", 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6622000000000001e-05 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344988+0000", 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6879999999999999e-06 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354864+0000", 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098759160000000002 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354938+0000", 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4179999999999998e-05 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355250+0000", 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000311973 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.361925+0000", 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066749859999999999 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362285+0000", 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036007999999999998 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362462+0000", 2024-09-16T14:07:26.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017646200000000001 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362523+0000", 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1151 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:26.681 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2550143600000006, 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017278714000000001, 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1151 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:26.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345442+0000", 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.42e-06 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345450+0000", 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2420000000000003e-06 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345461+0000", 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1395e-05 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345467+0000", 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3600000000000001e-06 2024-09-16T14:07:26.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355848+0000", 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010380162 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355940+0000", 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2629999999999999e-05 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356352+0000", 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041188100000000001 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362037+0000", 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005684911 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362545+0000", 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050792899999999995 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362636+0000", 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1479999999999998e-05 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362720+0000", 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3304000000000001e-05 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1154 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.252633135, 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015133097999999999, 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1154 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347823+0000", 2024-09-16T14:07:26.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347832+0000", 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347854+0000", 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2086000000000002e-05 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347864+0000", 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356827+0000", 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089628060000000002 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356897+0000", 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0494000000000006e-05 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357171+0000", 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027356199999999998 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362741+0000", 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055696920000000002 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362867+0000", 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012610799999999999 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362922+0000", 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4911000000000003e-05 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362955+0000", 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3810999999999999e-05 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.693 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1155 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2523466639999992, 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014975097999999999, 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.694 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1155 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348110+0000", 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6700000000000002e-07 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348114+0000", 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2560000000000004e-06 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348129+0000", 2024-09-16T14:07:26.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4725e-05 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348134+0000", 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.152e-06 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357588+0000", 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094536949999999998 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357666+0000", 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8664000000000004e-05 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357994+0000", 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032747700000000001 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362762+0000", 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047681829999999996 2024-09-16T14:07:26.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363017+0000", 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025504500000000002 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363056+0000", 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9357000000000003e-05 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363084+0000", 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7577e-05 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1161 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2493740619999993, 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017258941999999999, 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1161 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:26.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351083+0000", 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2270000000000001e-06 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351089+0000", 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.178e-06 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351108+0000", 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9457e-05 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351116+0000", 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6499999999999996e-06 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360527+0000", 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094112959999999996 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360645+0000", 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011791299999999999 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.361269+0000", 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062395199999999999 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365240+0000", 2024-09-16T14:07:26.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039707470000000002 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368249+0000", 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030092000000000001 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368302+0000", 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3146999999999998e-05 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368340+0000", 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8175e-05 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1162 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2476463179999993, 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017877857, 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1162 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352810+0000", 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.234e-06 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352818+0000", 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352831+0000", 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2839e-05 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352838+0000", 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1790000000000002e-06 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363259+0000", 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010421616 2024-09-16T14:07:26.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363370+0000", 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111134 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363713+0000", 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034221700000000002 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.367148+0000", 2024-09-16T14:07:26.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034350029999999998 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370616+0000", 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034687419999999999 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370660+0000", 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3365000000000002e-05 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370687+0000", 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7121999999999999e-05 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1164 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2465702929999996, 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016868919, 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1164 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:26.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353886+0000", 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2130000000000001e-06 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353892+0000", 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1299999999999998e-06 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353903+0000", 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0821e-05 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353909+0000", 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2909999999999998e-06 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364201+0000", 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010292318 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364295+0000", 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3667999999999998e-05 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364657+0000", 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036279999999999998 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.367209+0000", 2024-09-16T14:07:26.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025513250000000001 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370697+0000", 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034881600000000001 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370729+0000", 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1977999999999998e-05 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370754+0000", 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5215000000000002e-05 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1166 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2443227090000004, 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014686309999999999, 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1166 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9400000000000004e-07 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356137+0000", 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.473e-06 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356161+0000", 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3968000000000002e-05 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356167+0000", 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5840000000000002e-06 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365282+0000", 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091153269999999995 2024-09-16T14:07:26.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365369+0000", 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7064000000000005e-05 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365709+0000", 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034009900000000001 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368388+0000", 2024-09-16T14:07:26.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002679393 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370762+0000", 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023732319999999999 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370793+0000", 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1550000000000001e-05 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370819+0000", 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5826e-05 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1325 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8807145419999998, 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014200757, 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:26.724 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1325 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:26.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719742+0000", 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3459999999999999e-06 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719750+0000", 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6569999999999997e-06 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719777+0000", 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7370000000000001e-05 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719785+0000", 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2190000000000004e-06 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728088+0000", 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083028860000000006 2024-09-16T14:07:26.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728229+0000", 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014092799999999999 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728704+0000", 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047427599999999998 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.729579+0000", 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00087516299999999998 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733865+0000", 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042860570000000002 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733907+0000", 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789999999999998e-05 2024-09-16T14:07:26.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733942+0000", 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5064999999999997e-05 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:26.730 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:26.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:07:26.732 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:26.733 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:26.726+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:26.832 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:26.826+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:26.847 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone6 from 0x987421b (overlap 0x987421b) to 0x8935518 2024-09-16T14:07:26.890 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:26.890 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:26.890 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:26.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:26.933 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:26.926+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:26.997 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:26.997 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:26.997 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:26.998 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:26.998 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:27.007 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:27.048 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:27.042+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:27.135 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:27.130+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:27.153 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:27.153 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:27.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:27.153 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:27.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1626 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.948843+0000", 2024-09-16T14:07:27.154 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.1974379959999997, 2024-09-16T14:07:27.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012206928000000001, 2024-09-16T14:07:27.154 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.154 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.155 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1626 2024-09-16T14:07:27.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.155 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948843+0000", 2024-09-16T14:07:27.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948843+0000", 2024-09-16T14:07:27.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948844+0000", 2024-09-16T14:07:27.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.24e-06 2024-09-16T14:07:27.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948851+0000", 2024-09-16T14:07:27.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6980000000000001e-06 2024-09-16T14:07:27.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948868+0000", 2024-09-16T14:07:27.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7090000000000001e-05 2024-09-16T14:07:27.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948877+0000", 2024-09-16T14:07:27.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5969999999999998e-06 2024-09-16T14:07:27.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.953553+0000", 2024-09-16T14:07:27.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046760450000000002 2024-09-16T14:07:27.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.953759+0000", 2024-09-16T14:07:27.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020577 2024-09-16T14:07:27.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.954525+0000", 2024-09-16T14:07:27.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076622599999999995 2024-09-16T14:07:27.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.956072+0000", 2024-09-16T14:07:27.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001546668 2024-09-16T14:07:27.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.960981+0000", 2024-09-16T14:07:27.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049095069999999996 2024-09-16T14:07:27.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961020+0000", 2024-09-16T14:07:27.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9072000000000001e-05 2024-09-16T14:07:27.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961050+0000", 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9014999999999999e-05 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1627 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.948991+0000", 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.197289756, 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012292723, 2024-09-16T14:07:27.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1627 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948991+0000", 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948991+0000", 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948993+0000", 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.792e-06 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949001+0000", 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5099999999999998e-06 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949013+0000", 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2213e-05 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949021+0000", 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-06 2024-09-16T14:07:27.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.956134+0000", 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071138130000000001 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.956273+0000", 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138837 2024-09-16T14:07:27.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.956754+0000", 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048040499999999999 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961176+0000", 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004422245 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961220+0000", 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4447000000000003e-05 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961260+0000", 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9407999999999999e-05 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961284+0000", 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3802999999999999e-05 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.174 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1628 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.949184+0000", 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.1970970360000006, 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012158031999999999, 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.175 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1628 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949184+0000", 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949184+0000", 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949184+0000", 2024-09-16T14:07:27.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9800000000000002e-07 2024-09-16T14:07:27.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949189+0000", 2024-09-16T14:07:27.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4460000000000003e-06 2024-09-16T14:07:27.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949197+0000", 2024-09-16T14:07:27.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0029999999999994e-06 2024-09-16T14:07:27.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949201+0000", 2024-09-16T14:07:27.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5600000000000004e-06 2024-09-16T14:07:27.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.957713+0000", 2024-09-16T14:07:27.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085113259999999996 2024-09-16T14:07:27.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.957958+0000", 2024-09-16T14:07:27.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024565500000000001 2024-09-16T14:07:27.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.958927+0000", 2024-09-16T14:07:27.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096870500000000004 2024-09-16T14:07:27.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961215+0000", 2024-09-16T14:07:27.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022879380000000002 2024-09-16T14:07:27.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961293+0000", 2024-09-16T14:07:27.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8213000000000005e-05 2024-09-16T14:07:27.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961316+0000", 2024-09-16T14:07:27.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3048999999999999e-05 2024-09-16T14:07:27.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961342+0000", 2024-09-16T14:07:27.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5239e-05 2024-09-16T14:07:27.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.186 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.187 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1629 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.187 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.949445+0000", 2024-09-16T14:07:27.187 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.1968359589999995, 2024-09-16T14:07:27.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014464497, 2024-09-16T14:07:27.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.189 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1629 2024-09-16T14:07:27.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.189 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949445+0000", 2024-09-16T14:07:27.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949445+0000", 2024-09-16T14:07:27.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949446+0000", 2024-09-16T14:07:27.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.838e-06 2024-09-16T14:07:27.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949458+0000", 2024-09-16T14:07:27.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1632e-05 2024-09-16T14:07:27.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949481+0000", 2024-09-16T14:07:27.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2734999999999998e-05 2024-09-16T14:07:27.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949493+0000", 2024-09-16T14:07:27.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2607e-05 2024-09-16T14:07:27.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.959822+0000", 2024-09-16T14:07:27.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010328445 2024-09-16T14:07:27.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.959945+0000", 2024-09-16T14:07:27.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012324600000000001 2024-09-16T14:07:27.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.960448+0000", 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050304699999999996 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963619+0000", 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031706949999999999 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963711+0000", 2024-09-16T14:07:27.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1856999999999998e-05 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963805+0000", 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4499000000000004e-05 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963909+0000", 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103896 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1630 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.949671+0000", 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.1966096159999999, 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015796713, 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1630 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949671+0000", 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949671+0000", 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949672+0000", 2024-09-16T14:07:27.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949681+0000", 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9029999999999993e-06 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949704+0000", 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2994e-05 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949717+0000", 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3093e-05 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961385+0000", 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011667903 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961443+0000", 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7669e-05 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.962018+0000", 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00057456200000000003 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963628+0000", 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001610834 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965401+0000", 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001772643 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965436+0000", 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5311999999999997e-05 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965468+0000", 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1282999999999998e-05 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1631 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.949962+0000", 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.1963182890000006, 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015650797000000001, 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.205 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1631 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949962+0000", 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949962+0000", 2024-09-16T14:07:27.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949964+0000", 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949971+0000", 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6399999999999997e-06 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949999+0000", 2024-09-16T14:07:27.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8070999999999999e-05 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.950008+0000", 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7860000000000002e-06 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.962619+0000", 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012610974 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.962728+0000", 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108652 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963208+0000", 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048047500000000001 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.964241+0000", 2024-09-16T14:07:27.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010331819999999999 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965528+0000", 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012866520000000001 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965566+0000", 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7765000000000002e-05 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965613+0000", 2024-09-16T14:07:27.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7348000000000001e-05 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1665 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.000172+0000", 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.1461090110000001, 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012564687999999999, 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1665 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000172+0000", 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000172+0000", 2024-09-16T14:07:27.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000172+0000", 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000176+0000", 2024-09-16T14:07:27.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000185+0000", 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000191+0000", 2024-09-16T14:07:27.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8000000000000004e-06 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.009428+0000", 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092370690000000005 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.009596+0000", 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718200000000001 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.010079+0000", 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048326100000000002 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011893+0000", 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018137299999999999 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.012329+0000", 2024-09-16T14:07:27.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000436239 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.012647+0000", 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031829999999999998 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.012736+0000", 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9210999999999997e-05 2024-09-16T14:07:27.217 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1816 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.046846+0000", 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0994343460000007, 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010728168, 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1816 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046846+0000", 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046846+0000", 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046847+0000", 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2399999999999996e-07 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046853+0000", 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3600000000000004e-06 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046866+0000", 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3706000000000001e-05 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046878+0000", 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1479e-05 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.050825+0000", 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039476169999999996 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.050953+0000", 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012753299999999999 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.051482+0000", 2024-09-16T14:07:27.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052885900000000001 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.055718+0000", 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042367899999999998 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057381+0000", 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016628019999999999 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057474+0000", 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3083000000000003e-05 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057574+0000", 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100015 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1817 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.046886+0000", 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0993941809999992, 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011018309, 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1817 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.225 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046886+0000", 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046886+0000", 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046887+0000", 2024-09-16T14:07:27.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2200000000000003e-07 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046892+0000", 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3720000000000004e-06 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046903+0000", 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.063e-05 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046919+0000", 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.541e-05 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.052121+0000", 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052027150000000001 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.052263+0000", 2024-09-16T14:07:27.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014202400000000001 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.053046+0000", 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078279300000000005 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057303+0000", 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042572260000000002 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057767+0000", 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046386499999999999 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057857+0000", 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9645e-05 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057905+0000", 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7907000000000001e-05 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1818 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.046939+0000", 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0993415140000007, 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014369856, 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1818 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046939+0000", 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046939+0000", 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046940+0000", 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046945+0000", 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5339999999999999e-06 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046964+0000", 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8972000000000001e-05 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046973+0000", 2024-09-16T14:07:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.054337+0000", 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073648450000000001 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.054501+0000", 2024-09-16T14:07:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016394099999999999 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.055062+0000", 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056039299999999996 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057928+0000", 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028667160000000001 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061233+0000", 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033044250000000002 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061276+0000", 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2793999999999997e-05 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061309+0000", 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3253000000000002e-05 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1819 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.046981+0000", 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0992998790000001, 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014748755000000001, 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1819 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046981+0000", 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046981+0000", 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046981+0000", 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8599999999999997e-07 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046986+0000", 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8239999999999999e-06 2024-09-16T14:07:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046997+0000", 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.047e-05 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047004+0000", 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8759999999999999e-06 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.055780+0000", 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0087759699999999993 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.055880+0000", 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9982000000000006e-05 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.056480+0000", 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00060014900000000002 2024-09-16T14:07:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061135+0000", 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046551320000000002 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061429+0000", 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029431900000000003 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061614+0000", 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018478599999999999 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061729+0000", 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115461 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1820 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.047012+0000", 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0992685069999997, 2024-09-16T14:07:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014964712999999999, 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1820 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047012+0000", 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047012+0000", 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047013+0000", 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9299999999999997e-07 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047017+0000", 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4100000000000001e-06 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047037+0000", 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9712e-05 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047054+0000", 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7099999999999999e-05 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057980+0000", 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010925727999999999 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.058088+0000", 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010812800000000001 2024-09-16T14:07:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.058703+0000", 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00061499700000000005 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061757+0000", 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030545260000000001 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061846+0000", 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8480000000000007e-05 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061918+0000", 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2336000000000007e-05 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061977+0000", 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8603000000000001e-05 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1821 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.047069+0000", 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0992112449999993, 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016629976000000001, 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1821 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047069+0000", 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047069+0000", 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047070+0000", 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047075+0000", 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2820000000000004e-06 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047091+0000", 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.509e-05 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047097+0000", 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.059396+0000", 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012298953 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.059645+0000", 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000248409 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.060393+0000", 2024-09-16T14:07:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074888199999999998 2024-09-16T14:07:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.062007+0000", 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001613401 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.063631+0000", 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016242559999999999 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.063668+0000", 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7452e-05 2024-09-16T14:07:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.063699+0000", 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0811e-05 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3899 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:20.464653+0000", 2024-09-16T14:07:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 6.681627486, 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012489808999999999, 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3899 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464653+0000", 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464653+0000", 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464654+0000", 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464657+0000", 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3359999999999999e-06 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464662+0000", 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2480000000000004e-06 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464666+0000", 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.365e-06 2024-09-16T14:07:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473008+0000", 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083424339999999993 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473102+0000", 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4099999999999997e-05 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473371+0000", 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026890699999999998 2024-09-16T14:07:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.474455+0000", 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010837539999999999 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477086+0000", 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026313489999999998 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477121+0000", 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4785e-05 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477143+0000", 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1960999999999999e-05 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3900 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:20.464669+0000", 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 6.6816113689999996, 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012841968, 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3900 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464669+0000", 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464669+0000", 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464670+0000", 2024-09-16T14:07:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464672+0000", 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6579999999999998e-06 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464677+0000", 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1749999999999996e-06 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464681+0000", 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5109999999999999e-06 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.474531+0000", 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098500870000000004 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.474778+0000", 2024-09-16T14:07:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000247431 2024-09-16T14:07:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.475878+0000", 2024-09-16T14:07:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010996739999999999 2024-09-16T14:07:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477364+0000", 2024-09-16T14:07:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014854460000000001 2024-09-16T14:07:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477458+0000", 2024-09-16T14:07:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4901000000000005e-05 2024-09-16T14:07:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477489+0000", 2024-09-16T14:07:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0892000000000002e-05 2024-09-16T14:07:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477511+0000", 2024-09-16T14:07:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1810999999999998e-05 2024-09-16T14:07:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7542 3.18 3:1cd9c7f6:::rbd_data.11369af4c40c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.804895+0000", 2024-09-16T14:07:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3413856299999996, 2024-09-16T14:07:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017019457000000002, 2024-09-16T14:07:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7542 2024-09-16T14:07:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.804895+0000", 2024-09-16T14:07:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.804895+0000", 2024-09-16T14:07:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.804896+0000", 2024-09-16T14:07:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0890000000000001e-06 2024-09-16T14:07:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808057+0000", 2024-09-16T14:07:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031613840000000002 2024-09-16T14:07:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808074+0000", 2024-09-16T14:07:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:07:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808080+0000", 2024-09-16T14:07:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2889999999999997e-06 2024-09-16T14:07:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808145+0000", 2024-09-16T14:07:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5693999999999998e-05 2024-09-16T14:07:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808408+0000", 2024-09-16T14:07:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026258400000000001 2024-09-16T14:07:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809480+0000", 2024-09-16T14:07:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010722640000000001 2024-09-16T14:07:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821494+0000", 2024-09-16T14:07:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012013249 2024-09-16T14:07:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821798+0000", 2024-09-16T14:07:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000304701 2024-09-16T14:07:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821864+0000", 2024-09-16T14:07:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6214e-05 2024-09-16T14:07:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821914+0000", 2024-09-16T14:07:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9917000000000003e-05 2024-09-16T14:07:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7544 3.1f 3:f86efcdd:::rbd_data.11369af4c40c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.808092+0000", 2024-09-16T14:07:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3381887719999996, 2024-09-16T14:07:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014525727, 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7544 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808092+0000", 2024-09-16T14:07:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808092+0000", 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808093+0000", 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6799999999999999e-07 2024-09-16T14:07:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808098+0000", 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.383e-06 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808105+0000", 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.883e-06 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808108+0000", 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9909999999999998e-06 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808162+0000", 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4151999999999999e-05 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808339+0000", 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017649699999999999 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808637+0000", 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029818899999999997 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809503+0000", 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086654600000000005 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822551+0000", 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013047303 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822588+0000", 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7673000000000003e-05 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822618+0000", 2024-09-16T14:07:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9241999999999999e-05 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:7540 3.10 e35/23 3:0c0a7e4b:::rbd_data.11369af4c40c.0000000000000002:head v 35'40, mlcod=35'37)", 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.808130+0000", 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3381503749999997, 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013418888, 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808130+0000", 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808130+0000", 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808131+0000", 2024-09-16T14:07:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T14:07:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809736+0000", 2024-09-16T14:07:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001604865 2024-09-16T14:07:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809754+0000", 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8057999999999999e-05 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809772+0000", 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8266000000000001e-05 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809817+0000", 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5302000000000001e-05 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809839+0000", 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.213e-05 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821498+0000", 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011658577999999999 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821549+0000", 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1196999999999998e-05 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7549 3.1f 3:fc2cd15a:::rbd_data.11369af4c40c.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.808781+0000", 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3374997000000004, 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016760067, 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7549 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808781+0000", 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808781+0000", 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808782+0000", 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1500000000000003e-07 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811642+0000", 2024-09-16T14:07:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028604630000000002 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811659+0000", 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7022000000000001e-05 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811667+0000", 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6599999999999995e-06 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811769+0000", 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010229399999999999 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.812261+0000", 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049149199999999995 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814009+0000", 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017488040000000001 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.823432+0000", 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094223780000000004 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825298+0000", 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018666539999999999 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825439+0000", 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014035799999999999 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825541+0000", 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010212700000000001 2024-09-16T14:07:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7550 3.0 3:069bb4b1:::rbd_data.11369af4c40c.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.811687+0000", 2024-09-16T14:07:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3345931359999996, 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013961497999999999, 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7550 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811687+0000", 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811687+0000", 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811689+0000", 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7540000000000001e-06 2024-09-16T14:07:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814429+0000", 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00274011 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814448+0000", 2024-09-16T14:07:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9156000000000001e-05 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814455+0000", 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0060000000000003e-06 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814492+0000", 2024-09-16T14:07:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6952999999999997e-05 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814629+0000", 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136475 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.815396+0000", 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076686799999999998 2024-09-16T14:07:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822693+0000", 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072970450000000003 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825554+0000", 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028615670000000002 2024-09-16T14:07:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825595+0000", 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1044999999999998e-05 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825649+0000", 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3519e-05 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.315 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:27.317 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:27.318 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-16T14:07:27.318 INFO:teuthology.orchestra.run.smithi079.stdout:791 read 0x67df788 thru 0x67e5cbe (0x6537 bytes) 2024-09-16T14:07:27.318 INFO:teuthology.orchestra.run.smithi079.stdout:792 write 0xbf2dad1 thru 0xbf3b363 (0xd893 bytes) 2024-09-16T14:07:27.318 INFO:teuthology.orchestra.run.smithi079.stdout:793 read 0x9da283a thru 0x9daae01 (0x85c8 bytes) 2024-09-16T14:07:27.318 INFO:teuthology.orchestra.run.smithi079.stdout:794 write 0x2647df2 thru 0x264cd4c (0x4f5b bytes) 2024-09-16T14:07:27.318 INFO:teuthology.orchestra.run.smithi079.stdout:796 trunc from 0xbf3b364 to 0xdb76179 2024-09-16T14:07:27.318 INFO:teuthology.orchestra.run.smithi079.stdout:797 writesame 0xbea71 thru 0xce7e1 data_size 0x59(0xfd71 bytes) 2024-09-16T14:07:27.318 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:27.310+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:27.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:27.338 INFO:teuthology.orchestra.run.smithi079.stdout:799 read 0x5e914c thru 0x5f5351 (0xc206 bytes) 2024-09-16T14:07:27.352 INFO:teuthology.orchestra.run.smithi079.stdout:800 trunc from 0xdb76179 to 0x84903da 2024-09-16T14:07:27.406 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.416 INFO:teuthology.orchestra.run.smithi079.stdout:802 trunc from 0x84903da to 0x1ecb993 2024-09-16T14:07:27.417 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:27.410+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:27.421 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:27.421 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:27.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.421 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9880 3.12 3:4adb2a6c:::rbd_data.1136a719415e.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:27.421 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.417201+0000", 2024-09-16T14:07:27.421 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0018103259999999999, 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018840809999999999, 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9880 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417201+0000", 2024-09-16T14:07:27.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417201+0000", 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417203+0000", 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.903e-06 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417213+0000", 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9499999999999996e-06 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417246+0000", 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2851000000000001e-05 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:27.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417258+0000", 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.203e-05 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417346+0000", 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7967999999999995e-05 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:27.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417498+0000", 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152103 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.417656+0000", 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158209 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.418529+0000", 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000872871 2024-09-16T14:07:27.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.427 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:27.427 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.427 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:27.427 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:07:27.427 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T14:07:27.427 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:27.431 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:27.438 INFO:teuthology.orchestra.run.smithi079.stdout:803 read 0x62dcb6 thru 0x632b3e (0x4e89 bytes) 2024-09-16T14:07:27.455 INFO:teuthology.orchestra.run.smithi079.stdout:804 writesame 0x751eb33 thru 0x751edca data_size 0x4(0x298 bytes) 2024-09-16T14:07:27.455 INFO:teuthology.orchestra.run.smithi079.stdout:805 trunc from 0x751edcb to 0x71c3acc 2024-09-16T14:07:27.461 INFO:teuthology.orchestra.run.smithi079.stdout:807 read 0x2eb00cb thru 0x2eb1652 (0x1588 bytes) 2024-09-16T14:07:27.462 INFO:teuthology.orchestra.run.smithi079.stdout:808 punch from 0x69f1caf to 0x6a00056, (0xe3a7 bytes) 2024-09-16T14:07:27.462 INFO:teuthology.orchestra.run.smithi079.stdout:809 writesame 0x34842de thru 0x348ffcf data_size 0x46(0xbcf2 bytes) 2024-09-16T14:07:27.464 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:27.482 INFO:teuthology.orchestra.run.smithi079.stdout:810 writesame 0xadabdc6 thru 0xadb0dab data_size 0xe(0x4fe6 bytes) 2024-09-16T14:07:27.483 INFO:teuthology.orchestra.run.smithi079.stdout:811 punch from 0x838cda8 to 0x839743c, (0xa694 bytes) 2024-09-16T14:07:27.483 INFO:teuthology.orchestra.run.smithi079.stdout:812 read 0x721259a thru 0x721f44d (0xceb4 bytes) 2024-09-16T14:07:27.483 INFO:teuthology.orchestra.run.smithi079.stdout:814 punch from 0x7d4bf75 to 0x7d4f440, (0x34cb bytes) 2024-09-16T14:07:27.489 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:27.495 INFO:teuthology.orchestra.run.smithi079.stdout:815 write 0xe3782f9 thru 0xe37c2d4 (0x3fdc bytes) 2024-09-16T14:07:27.501 INFO:teuthology.orchestra.run.smithi079.stdout:816 read 0x7d0c0a8 thru 0x7d0c3f1 (0x34a bytes) 2024-09-16T14:07:27.501 INFO:teuthology.orchestra.run.smithi079.stdout:817 writesame 0x8aec626 thru 0x8af79c6 data_size 0x11(0xb3a1 bytes) 2024-09-16T14:07:27.506 INFO:teuthology.orchestra.run.smithi079.stdout:820 write 0x8e70ae9 thru 0x8e76d0c (0x6224 bytes) 2024-09-16T14:07:27.511 INFO:teuthology.orchestra.run.smithi079.stdout:821 write 0x2d2280c thru 0x2d24a22 (0x2217 bytes) 2024-09-16T14:07:27.514 INFO:teuthology.orchestra.run.smithi079.stdout:822 write 0x71ec06e thru 0x71fb7b8 (0xf74b bytes) 2024-09-16T14:07:27.517 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:27.518 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:27.510+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:27.520 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.520 INFO:teuthology.orchestra.run.smithi079.stdout:826 writesame 0x66cc758 thru 0x66dc4ab data_size 0x1f(0xfd54 bytes) 2024-09-16T14:07:27.526 INFO:teuthology.orchestra.run.smithi079.stdout:828 punch from 0x356eb5c to 0x3570815, (0x1cb9 bytes) 2024-09-16T14:07:27.526 INFO:teuthology.orchestra.run.smithi079.stdout:829 write 0xfd10ef thru 0xfd1105 (0x17 bytes) 2024-09-16T14:07:27.527 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:27.528 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:27.530 INFO:teuthology.orchestra.run.smithi079.stdout:830 read 0xd9a3ef4 thru 0xd9a7d09 (0x3e16 bytes) 2024-09-16T14:07:27.530 INFO:teuthology.orchestra.run.smithi079.stdout:831 write 0xd4beac2 thru 0xd4c0b90 (0x20cf bytes) 2024-09-16T14:07:27.535 INFO:teuthology.orchestra.run.smithi079.stdout:832 writesame 0x2ece599 thru 0x2edd229 data_size 0x9(0xec91 bytes) 2024-09-16T14:07:27.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:27.537 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:27.543 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.544 INFO:teuthology.orchestra.run.smithi079.stdout:836 write 0x1543a5f thru 0x154d453 (0x99f5 bytes) 2024-09-16T14:07:27.547 INFO:teuthology.orchestra.run.smithi079.stdout:837 write 0xbb0a468 thru 0xbb0f4f3 (0x508c bytes) 2024-09-16T14:07:27.559 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:27.571 INFO:teuthology.orchestra.run.smithi079.stdout:838 trunc from 0xe37c2d5 to 0x51a29d1 2024-09-16T14:07:27.582 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:27.588 INFO:teuthology.orchestra.run.smithi079.stdout:841 read 0x388e64 thru 0x38d5c1 (0x475e bytes) 2024-09-16T14:07:27.589 INFO:teuthology.orchestra.run.smithi079.stdout:842 read 0x2709512 thru 0x270b19a (0x1c89 bytes) 2024-09-16T14:07:27.590 INFO:teuthology.orchestra.run.smithi079.stdout:843 read 0x1fda926 thru 0x1fe41d9 (0x98b4 bytes) 2024-09-16T14:07:27.591 INFO:teuthology.orchestra.run.smithi079.stdout:844 punch from 0x37749e4 to 0x377a653, (0x5c6f bytes) 2024-09-16T14:07:27.598 INFO:teuthology.orchestra.run.smithi079.stdout:845 trunc from 0x51a29d1 to 0x8a703ed 2024-09-16T14:07:27.605 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:27.606 INFO:teuthology.orchestra.run.smithi079.stdout:846 write 0x786c403 thru 0x78755bc (0x91ba bytes) 2024-09-16T14:07:27.618 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:27.614+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:27.623 INFO:teuthology.orchestra.run.smithi079.stdout:847 write 0xacaea5a thru 0xacb71d8 (0x877f bytes) 2024-09-16T14:07:27.634 INFO:teuthology.orchestra.run.smithi079.stdout:848 write 0xad2ea10 thru 0xad3924b (0xa83c bytes) 2024-09-16T14:07:27.634 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:27.636 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:27.636 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:27.637 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:27.637 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:27.637 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:27.637 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:27.637 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:27.637 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:27.639 INFO:teuthology.orchestra.run.smithi079.stdout:849 writesame 0x1ce3155 thru 0x1ce7856 data_size 0x2(0x4702 bytes) 2024-09-16T14:07:27.648 INFO:teuthology.orchestra.run.smithi079.stdout:850 punch from 0x1858fb8 to 0x1866d4b, (0xdd93 bytes) 2024-09-16T14:07:27.664 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:07:27.665 INFO:teuthology.orchestra.run.smithi079.stdout:851 trunc from 0xad3924c to 0xda663c 2024-09-16T14:07:27.683 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:27.683 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:27.692 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:27.699 INFO:teuthology.orchestra.run.smithi079.stdout:852 write 0x1566653 thru 0x156e176 (0x7b24 bytes) 2024-09-16T14:07:27.701 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.720 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:27.714+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:27.728 INFO:teuthology.orchestra.run.smithi079.stdout:855 writesame 0xb7d76c3 thru 0xb7e314d data_size 0x1(0xba8b bytes) 2024-09-16T14:07:27.728 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.729 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.729 INFO:teuthology.orchestra.run.smithi079.stdout:859 read 0x4e3598c thru 0x4e441a6 (0xe81b bytes) 2024-09-16T14:07:27.739 INFO:teuthology.orchestra.run.smithi079.stdout:860 write 0xbcc9cb2 thru 0xbcccf20 (0x326f bytes) 2024-09-16T14:07:27.756 INFO:teuthology.orchestra.run.smithi079.stdout:861 trunc from 0xbcccf21 to 0x5833836 2024-09-16T14:07:27.776 INFO:teuthology.orchestra.run.smithi079.stdout:863 read 0xbb996d thru 0xbbc660 (0x2cf4 bytes) 2024-09-16T14:07:27.777 INFO:teuthology.orchestra.run.smithi079.stdout:865 punch from 0x1752281 to 0x1762155, (0xfed4 bytes) 2024-09-16T14:07:27.777 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.777 INFO:teuthology.orchestra.run.smithi079.stdout:868 writesame 0x3cec29b thru 0x3cec2e7 data_size 0x1(0x4d bytes) 2024-09-16T14:07:27.783 INFO:teuthology.orchestra.run.smithi079.stdout:869 punch from 0x4161bdc to 0x416ebb8, (0xcfdc bytes) 2024-09-16T14:07:27.784 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.795 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:27.795 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:27.801 INFO:teuthology.orchestra.run.smithi079.stdout:872 writesame 0x8a08fb9 thru 0x8a0e06b data_size 0x1(0x50b3 bytes) 2024-09-16T14:07:27.801 INFO:teuthology.orchestra.run.smithi079.stdout:874 read 0x7d4b037 thru 0x7d515eb (0x65b5 bytes) 2024-09-16T14:07:27.801 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.803 INFO:teuthology.orchestra.run.smithi079.stdout:876 trunc from 0x8a0e06c to 0x809386f 2024-09-16T14:07:27.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:27.818 INFO:teuthology.orchestra.run.smithi079.stdout:878 trunc from 0x809386f to 0x5f428c7 2024-09-16T14:07:27.820 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:27.814+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:27.827 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:27.828 INFO:teuthology.orchestra.run.smithi079.stdout:880 punch from 0x4420e6b to 0x44211a5, (0x33a bytes) 2024-09-16T14:07:27.828 INFO:teuthology.orchestra.run.smithi079.stdout:881 read 0x5bc31df thru 0x5bd192a (0xe74c bytes) 2024-09-16T14:07:27.838 INFO:teuthology.orchestra.run.smithi079.stdout:882 write 0x75db84c thru 0x75df511 (0x3cc6 bytes) 2024-09-16T14:07:27.844 INFO:teuthology.orchestra.run.smithi079.stdout:883 write 0xe3ad2c thru 0xe3b6d4 (0x9a9 bytes) 2024-09-16T14:07:27.863 INFO:teuthology.orchestra.run.smithi079.stdout:884 write 0xa544ce0 thru 0xa5532fc (0xe61d bytes) 2024-09-16T14:07:27.873 INFO:teuthology.orchestra.run.smithi079.stdout:885 trunc from 0xa5532fd to 0x8b3c5ae 2024-09-16T14:07:27.903 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:27.903 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:27.907 INFO:teuthology.orchestra.run.smithi079.stdout:886 trunc from 0x8b3c5ae to 0x1f7476b 2024-09-16T14:07:27.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:27.921 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:27.914+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:27.953 INFO:teuthology.orchestra.run.smithi079.stdout:887 write 0xcb967d0 thru 0xcb9a375 (0x3ba6 bytes) 2024-09-16T14:07:27.973 INFO:teuthology.orchestra.run.smithi079.stdout:888 trunc from 0xcb9a376 to 0x4271bc8 2024-09-16T14:07:27.988 INFO:teuthology.orchestra.run.smithi079.stdout:889 read 0x33e4057 thru 0x33ed210 (0x91ba bytes) 2024-09-16T14:07:28.009 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:28.010 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:28.013 INFO:teuthology.orchestra.run.smithi079.stdout:890 write 0xd89588b thru 0xd89efbc (0x9732 bytes) 2024-09-16T14:07:28.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:28.019 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:28.028 INFO:teuthology.orchestra.run.smithi079.stdout:893 write 0xe0bd886 thru 0xe0cbd2b (0xe4a6 bytes) 2024-09-16T14:07:28.029 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:28.022+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:28.035 INFO:teuthology.orchestra.run.smithi079.stdout:896 writesame 0xcc6062b thru 0xcc60756 data_size 0xc(0x12c bytes) 2024-09-16T14:07:28.040 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:28.040 INFO:teuthology.orchestra.run.smithi079.stdout:898 read 0x1359949 thru 0x1368251 (0xe909 bytes) 2024-09-16T14:07:28.041 INFO:teuthology.orchestra.run.smithi079.stdout:899 write 0x977c9a5 thru 0x9780f54 (0x45b0 bytes) 2024-09-16T14:07:28.048 INFO:teuthology.orchestra.run.smithi079.stdout:901 write 0xdca95d3 thru 0xdcb7750 (0xe17e bytes) 2024-09-16T14:07:28.054 INFO:teuthology.orchestra.run.smithi079.stdout:902 write 0x43e0a2b thru 0x43e48fb (0x3ed1 bytes) 2024-09-16T14:07:28.059 INFO:teuthology.orchestra.run.smithi079.stdout:905 write 0x4ce3d03 thru 0x4ce7f56 (0x4254 bytes) 2024-09-16T14:07:28.065 INFO:teuthology.orchestra.run.smithi079.stdout:906 write 0x6f717ef thru 0x6f7b425 (0x9c37 bytes) 2024-09-16T14:07:28.071 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:28.071 INFO:teuthology.orchestra.run.smithi079.stdout:909 write 0x9fbe25 thru 0x9fc256 (0x432 bytes) 2024-09-16T14:07:28.075 INFO:teuthology.orchestra.run.smithi079.stdout:911 write 0x3581011 thru 0x35880c6 (0x70b6 bytes) 2024-09-16T14:07:28.093 INFO:teuthology.orchestra.run.smithi079.stdout:912 write 0xed19a63 thru 0xed23f16 (0xa4b4 bytes) 2024-09-16T14:07:28.096 INFO:teuthology.orchestra.run.smithi079.stdout:913 write 0xfcc56e thru 0xfd401c (0x7aaf bytes) 2024-09-16T14:07:28.100 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:28.100 INFO:teuthology.orchestra.run.smithi079.stdout:915 write 0xc936c4a thru 0xc940550 (0x9907 bytes) 2024-09-16T14:07:28.103 INFO:teuthology.orchestra.run.smithi079.stdout:918 read 0xa76c654 thru 0xa76ef9c (0x2949 bytes) 2024-09-16T14:07:28.104 INFO:teuthology.orchestra.run.smithi079.stdout:919 writesame 0x158cea3 thru 0x159bc06 data_size 0x2(0xed64 bytes) 2024-09-16T14:07:28.122 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:28.122 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:28.122 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:28.114+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:28.128 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:28.129 INFO:teuthology.orchestra.run.smithi079.stdout:921 write 0x68f1fbc thru 0x68fa98f (0x89d4 bytes) 2024-09-16T14:07:28.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:28.136 INFO:teuthology.orchestra.run.smithi079.stdout:922 clone 8 order 23 su 1048576 sc 5 2024-09-16T14:07:28.223 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:28.218+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:28.232 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:28.232 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:28.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:28.324 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:28.318+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:28.342 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:28.342 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:28.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:28.424 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:28.418+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:28.452 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:28.452 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:28.462 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:28.526 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:28.522+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:28.610 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:28.611 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:28.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:28.627 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:28.618+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:28.728 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:28.722+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:28.733 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:28.733 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:28.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:28.828 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:28.822+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:28.846 INFO:teuthology.orchestra.run.smithi079.stdout:leaving image image_client.0-clone7 intact 2024-09-16T14:07:28.893 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:28.893 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:28.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:28.929 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:28.926+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:29.009 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.010 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.019 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:29.030 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:29.022+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:29.122 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.122 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:29.132 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:29.126+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:29.221 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-16T14:07:29.232 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:29.226+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:29.237 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.237 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:29.333 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:29.326+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:29.379 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.379 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:29.442 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:29.438+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:29.503 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.503 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.512 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:29.535 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:29.530+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:29.599 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.600 INFO:teuthology.orchestra.run.smithi079.stdout:924 write 0x7bde9d8 thru 0x7bedc61 (0xf28a bytes) 2024-09-16T14:07:29.614 INFO:teuthology.orchestra.run.smithi079.stdout:925 writesame 0x718a3cd thru 0x7198a9e data_size 0x26(0xe6d2 bytes) 2024-09-16T14:07:29.620 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.620 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.629 INFO:teuthology.orchestra.run.smithi079.stdout:926 trunc from 0xed23f17 to 0xaea9253 2024-09-16T14:07:29.630 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:29.635 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:29.630+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:29.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:29.665 INFO:teuthology.orchestra.run.smithi079.stdout:927 write 0xe072d28 thru 0xe0826ab (0xf984 bytes) 2024-09-16T14:07:29.671 INFO:teuthology.orchestra.run.smithi079.stdout:928 read 0x92c94be thru 0x92ce794 (0x52d7 bytes) 2024-09-16T14:07:29.671 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.671 INFO:teuthology.orchestra.run.smithi079.stdout:930 write 0xd03c8df thru 0xd041061 (0x4783 bytes) 2024-09-16T14:07:29.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:07:29.678 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.678 INFO:teuthology.orchestra.run.smithi079.stdout:932 write 0x4eb66d6 thru 0x4eb8020 (0x194b bytes) 2024-09-16T14:07:29.685 INFO:teuthology.orchestra.run.smithi079.stdout:933 write 0x1c2a7d0 thru 0x1c305c8 (0x5df9 bytes) 2024-09-16T14:07:29.694 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.694 INFO:teuthology.orchestra.run.smithi079.stdout:935 read 0x56c912f thru 0x56d476c (0xb63e bytes) 2024-09-16T14:07:29.694 INFO:teuthology.orchestra.run.smithi079.stdout:937 write 0xc152e32 thru 0xc156535 (0x3704 bytes) 2024-09-16T14:07:29.696 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.697 INFO:teuthology.orchestra.run.smithi079.stdout:939 write 0x80497e8 thru 0x8053de0 (0xa5f9 bytes) 2024-09-16T14:07:29.701 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.701 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.701 INFO:teuthology.orchestra.run.smithi079.stdout:945 write 0x8a85ca3 thru 0x8a8d33e (0x769c bytes) 2024-09-16T14:07:29.703 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.703 INFO:teuthology.orchestra.run.smithi079.stdout:950 write 0x21a3d14 thru 0x21a8984 (0x4c71 bytes) 2024-09-16T14:07:29.705 INFO:teuthology.orchestra.run.smithi079.stdout:951 punch from 0xc407712 to 0xc40ffbb, (0x88a9 bytes) 2024-09-16T14:07:29.705 INFO:teuthology.orchestra.run.smithi079.stdout:953 write 0xcb593d7 thru 0xcb675e2 (0xe20c bytes) 2024-09-16T14:07:29.716 INFO:teuthology.orchestra.run.smithi079.stdout:954 trunc from 0xe0826ac to 0x8d1a9c5 2024-09-16T14:07:29.736 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.736 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.736 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:29.730+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:29.745 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:29.749 INFO:teuthology.orchestra.run.smithi079.stdout:955 write 0x9c3308d thru 0x9c40adb (0xda4f bytes) 2024-09-16T14:07:29.766 INFO:teuthology.orchestra.run.smithi079.stdout:956 write 0xe743ebc thru 0xe745caa (0x1def bytes) 2024-09-16T14:07:29.771 INFO:teuthology.orchestra.run.smithi079.stdout:961 write 0x5dabb30 thru 0x5db009e (0x456f bytes) 2024-09-16T14:07:29.789 INFO:teuthology.orchestra.run.smithi079.stdout:962 trunc from 0xe745cab to 0x4d8a0d4 2024-09-16T14:07:29.794 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:29.812 INFO:teuthology.orchestra.run.smithi079.stdout:964 read 0x17456ce thru 0x1749c47 (0x457a bytes) 2024-09-16T14:07:29.827 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:29.831 INFO:teuthology.orchestra.run.smithi079.stdout:965 write 0xcae8038 thru 0xcaefa9c (0x7a65 bytes) 2024-09-16T14:07:29.837 INFO:teuthology.orchestra.run.smithi079.stdout:966 trunc from 0xcaefa9d to 0xd4632c8 2024-09-16T14:07:29.837 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:29.830+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:29.842 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.842 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.851 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:29.855 INFO:teuthology.orchestra.run.smithi079.stdout:967 trunc from 0xd4632c8 to 0x5c2b24d 2024-09-16T14:07:29.858 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:29.888 INFO:teuthology.orchestra.run.smithi079.stdout:968 write 0xd46d95d thru 0xd46ef9e (0x1642 bytes) 2024-09-16T14:07:29.890 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:29.892 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.892 INFO:teuthology.orchestra.run.smithi079.stdout:970 read 0x9cebe8 thru 0x9cf664 (0xa7d bytes) 2024-09-16T14:07:29.913 INFO:teuthology.orchestra.run.smithi079.stdout:971 trunc from 0xd46ef9f to 0x2812a07 2024-09-16T14:07:29.925 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:29.938 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:29.930+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:29.941 INFO:teuthology.orchestra.run.smithi079.stdout:973 punch from 0x18b0d4c to 0x18bf62a, (0xe8de bytes) 2024-09-16T14:07:29.941 INFO:teuthology.orchestra.run.smithi079.stdout:976 read 0x603f8a thru 0x60c904 (0x897b bytes) 2024-09-16T14:07:29.943 INFO:teuthology.orchestra.run.smithi079.stdout:977 trunc from 0x2812a07 to 0x33ce914 2024-09-16T14:07:29.949 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:29.949 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:29.958 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:29.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:29.961 INFO:teuthology.orchestra.run.smithi079.stdout:978 punch from 0x1d7c4fc to 0x1d83bdd, (0x76e1 bytes) 2024-09-16T14:07:29.971 INFO:teuthology.orchestra.run.smithi079.stdout:979 trunc from 0x33ce914 to 0x9f1cd60 2024-09-16T14:07:29.979 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:29.979 INFO:teuthology.orchestra.run.smithi079.stdout:981 read 0x5d1fe8d thru 0x5d22e74 (0x2fe8 bytes) 2024-09-16T14:07:29.979 INFO:teuthology.orchestra.run.smithi079.stdout:982 write 0x1f25f4a thru 0x1f3553a (0xf5f1 bytes) 2024-09-16T14:07:29.981 INFO:teuthology.orchestra.run.smithi079.stdout:984 read 0x46bbfc1 thru 0x46be10d (0x214d bytes) 2024-09-16T14:07:29.990 INFO:teuthology.orchestra.run.smithi079.stdout:985 write 0xa3a75f7 thru 0xa3a9d70 (0x277a bytes) 2024-09-16T14:07:29.990 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:29.995 INFO:teuthology.orchestra.run.smithi079.stdout:986 writesame 0x95c527a thru 0x95d3aa9 data_size 0x28(0xe830 bytes) 2024-09-16T14:07:30.001 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.001 INFO:teuthology.orchestra.run.smithi079.stdout:988 writesame 0x393a779 thru 0x393fdbd data_size 0x7(0x5645 bytes) 2024-09-16T14:07:30.008 INFO:teuthology.orchestra.run.smithi079.stdout:989 read 0x8ecf29a thru 0x8ed7594 (0x82fb bytes) 2024-09-16T14:07:30.008 INFO:teuthology.orchestra.run.smithi079.stdout:990 write 0x94f0bdc thru 0x94f6e49 (0x626e bytes) 2024-09-16T14:07:30.019 INFO:teuthology.orchestra.run.smithi079.stdout:991 trunc from 0xa3a9d71 to 0xdddf1af 2024-09-16T14:07:30.021 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:30.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:30.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:30.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:30.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:30.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:30.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:30.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:30.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:30.028 INFO:teuthology.orchestra.run.smithi079.stdout:992 punch from 0xa98e8c to 0xaa19e0, (0x8b54 bytes) 2024-09-16T14:07:30.028 INFO:teuthology.orchestra.run.smithi079.stdout:993 write 0x769f0b8 thru 0x76a6c1b (0x7b64 bytes) 2024-09-16T14:07:30.033 INFO:teuthology.orchestra.run.smithi079.stdout:994 read 0x143960a thru 0x143fadd (0x64d4 bytes) 2024-09-16T14:07:30.034 INFO:teuthology.orchestra.run.smithi079.stdout:995 punch from 0x7ca808b to 0x7cb1317, (0x928c bytes) 2024-09-16T14:07:30.034 INFO:teuthology.orchestra.run.smithi079.stdout:997 write 0x850e28e thru 0x850f9e6 (0x1759 bytes) 2024-09-16T14:07:30.036 INFO:teuthology.orchestra.run.smithi079.stdout:999 read 0x6c33cf7 thru 0x6c34270 (0x57a bytes) 2024-09-16T14:07:30.037 INFO:teuthology.orchestra.run.smithi079.stdout:1000 write 0xdbdb22e thru 0xdbdffd1 (0x4da4 bytes) 2024-09-16T14:07:30.039 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:30.034+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:30.042 INFO:teuthology.orchestra.run.smithi079.stdout:1002 write 0x34d3d09 thru 0x34db222 (0x751a bytes) 2024-09-16T14:07:30.046 INFO:teuthology.orchestra.run.smithi079.stdout:1004 punch from 0x7036705 to 0x703fd7f, (0x967a bytes) 2024-09-16T14:07:30.046 INFO:teuthology.orchestra.run.smithi079.stdout:1005 read 0xab7f2c thru 0xac0304 (0x83d9 bytes) 2024-09-16T14:07:30.053 INFO:teuthology.orchestra.run.smithi079.stdout:1010 write 0x4762c98 thru 0x47700d4 (0xd43d bytes) 2024-09-16T14:07:30.058 INFO:teuthology.orchestra.run.smithi079.stdout:1011 writesame 0xbd708e8 thru 0xbd78663 data_size 0x6(0x7d7c bytes) 2024-09-16T14:07:30.065 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.065 INFO:teuthology.orchestra.run.smithi079.stdout:1014 punch from 0x18c60ec to 0x18cd5d4, (0x74e8 bytes) 2024-09-16T14:07:30.065 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.065 INFO:teuthology.orchestra.run.smithi079.stdout:1017 write 0x348361e thru 0x348bc46 (0x8629 bytes) 2024-09-16T14:07:30.067 INFO:teuthology.orchestra.run.smithi079.stdout:1018 read 0x520e5cb thru 0x52143f3 (0x5e29 bytes) 2024-09-16T14:07:30.068 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.068 INFO:teuthology.orchestra.run.smithi079.stdout:1020 read 0x4ff1beb thru 0x4ffa78e (0x8ba4 bytes) 2024-09-16T14:07:30.068 INFO:teuthology.orchestra.run.smithi079.stdout:1021 read 0xca0bf6f thru 0xca1aa61 (0xeaf3 bytes) 2024-09-16T14:07:30.068 INFO:teuthology.orchestra.run.smithi079.stdout:1022 read 0x176ca0a thru 0x176e5ed (0x1be4 bytes) 2024-09-16T14:07:30.068 INFO:teuthology.orchestra.run.smithi079.stdout:1023 read 0x46f1663 thru 0x46f38f4 (0x2292 bytes) 2024-09-16T14:07:30.069 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.079 INFO:teuthology.orchestra.run.smithi079.stdout:1025 trunc from 0xdddf1af to 0x77ed011 2024-09-16T14:07:30.096 INFO:teuthology.orchestra.run.smithi079.stdout:1026 read 0x16ad6fd thru 0x16b8643 (0xaf47 bytes) 2024-09-16T14:07:30.096 INFO:teuthology.orchestra.run.smithi079.stdout:1028 writesame 0x69f51a5 thru 0x69fe062 data_size 0x16(0x8ebe bytes) 2024-09-16T14:07:30.102 INFO:teuthology.orchestra.run.smithi079.stdout:1029 write 0x6b7e4fd thru 0x6b85fad (0x7ab1 bytes) 2024-09-16T14:07:30.109 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:30.109 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:30.109 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:30.109 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:30.109 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:30.119 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:30.123 INFO:teuthology.orchestra.run.smithi079.stdout:1031 write 0xb99e205 thru 0xb9a1b11 (0x390d bytes) 2024-09-16T14:07:30.127 INFO:teuthology.orchestra.run.smithi079.stdout:1032 punch from 0x4a72a7 to 0x4aad09, (0x3a62 bytes) 2024-09-16T14:07:30.135 INFO:teuthology.orchestra.run.smithi079.stdout:1033 trunc from 0xb9a1b12 to 0x735b9a8 2024-09-16T14:07:30.140 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:30.134+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:30.156 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.156 INFO:teuthology.orchestra.run.smithi079.stdout:1035 punch from 0x21ef29d to 0x21fe27f, (0xefe2 bytes) 2024-09-16T14:07:30.156 INFO:teuthology.orchestra.run.smithi079.stdout:1037 read 0x61a2294 thru 0x61a2a61 (0x7ce bytes) 2024-09-16T14:07:30.156 INFO:teuthology.orchestra.run.smithi079.stdout:1038 read 0x2d035aa thru 0x2d0d859 (0xa2b0 bytes) 2024-09-16T14:07:30.158 INFO:teuthology.orchestra.run.smithi079.stdout:1039 write 0x707a5a thru 0x7115b1 (0x9b58 bytes) 2024-09-16T14:07:30.160 INFO:teuthology.orchestra.run.smithi079.stdout:1040 write 0x432ddb0 thru 0x432e9bb (0xc0c bytes) 2024-09-16T14:07:30.163 INFO:teuthology.orchestra.run.smithi079.stdout:1041 write 0x52e1c7e thru 0x52ec14f (0xa4d2 bytes) 2024-09-16T14:07:30.182 INFO:teuthology.orchestra.run.smithi079.stdout:1042 trunc from 0x735b9a8 to 0x257a46 2024-09-16T14:07:30.224 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:30.224 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1626 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.948843+0000", 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.269846462, 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012206928000000001, 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1626 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948843+0000", 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948843+0000", 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948844+0000", 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.24e-06 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948851+0000", 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6980000000000001e-06 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948868+0000", 2024-09-16T14:07:30.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7090000000000001e-05 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948877+0000", 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5969999999999998e-06 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.953553+0000", 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046760450000000002 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.953759+0000", 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020577 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.954525+0000", 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076622599999999995 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.956072+0000", 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001546668 2024-09-16T14:07:30.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.960981+0000", 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049095069999999996 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961020+0000", 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9072000000000001e-05 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961050+0000", 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9014999999999999e-05 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1627 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.948991+0000", 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.269698222000001, 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012292723, 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1627 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948991+0000", 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948991+0000", 2024-09-16T14:07:30.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.948993+0000", 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.792e-06 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949001+0000", 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5099999999999998e-06 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949013+0000", 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2213e-05 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949021+0000", 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.25e-06 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.956134+0000", 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071138130000000001 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.956273+0000", 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138837 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.956754+0000", 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048040499999999999 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961176+0000", 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004422245 2024-09-16T14:07:30.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961220+0000", 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4447000000000003e-05 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961260+0000", 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9407999999999999e-05 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961284+0000", 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3802999999999999e-05 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1628 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.949184+0000", 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.269505501999999, 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012158031999999999, 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1628 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949184+0000", 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949184+0000", 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949184+0000", 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9800000000000002e-07 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949189+0000", 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4460000000000003e-06 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949197+0000", 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0029999999999994e-06 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949201+0000", 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5600000000000004e-06 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.957713+0000", 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085113259999999996 2024-09-16T14:07:30.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.957958+0000", 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024565500000000001 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.958927+0000", 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096870500000000004 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961215+0000", 2024-09-16T14:07:30.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022879380000000002 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961293+0000", 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8213000000000005e-05 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961316+0000", 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3048999999999999e-05 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961342+0000", 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5239e-05 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1629 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.949445+0000", 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.269244425, 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014464497, 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.243 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1629 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949445+0000", 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949445+0000", 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949446+0000", 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.838e-06 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949458+0000", 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1632e-05 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949481+0000", 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2734999999999998e-05 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949493+0000", 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2607e-05 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.959822+0000", 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010328445 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.959945+0000", 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012324600000000001 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.960448+0000", 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050304699999999996 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963619+0000", 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031706949999999999 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963711+0000", 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1856999999999998e-05 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963805+0000", 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4499000000000004e-05 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963909+0000", 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103896 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1630 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.949671+0000", 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.269018082000001, 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015796713, 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1630 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949671+0000", 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949671+0000", 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949672+0000", 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T14:07:30.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949681+0000", 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9029999999999993e-06 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949704+0000", 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2994e-05 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949717+0000", 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3093e-05 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961385+0000", 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011667903 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.961443+0000", 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7669e-05 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.962018+0000", 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00057456200000000003 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963628+0000", 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001610834 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965401+0000", 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001772643 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965436+0000", 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5311999999999997e-05 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965468+0000", 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1282999999999998e-05 2024-09-16T14:07:30.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1631 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.949962+0000", 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.268726754999999, 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015650797000000001, 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1631 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949962+0000", 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949962+0000", 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949964+0000", 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949971+0000", 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6399999999999997e-06 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.949999+0000", 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8070999999999999e-05 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.950008+0000", 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7860000000000002e-06 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.962619+0000", 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012610974 2024-09-16T14:07:30.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.962728+0000", 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108652 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.963208+0000", 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048047500000000001 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.964241+0000", 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010331819999999999 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965528+0000", 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012866520000000001 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965566+0000", 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7765000000000002e-05 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.965613+0000", 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7348000000000001e-05 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1665 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.000172+0000", 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.218517477000001, 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012564687999999999, 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.262 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1665 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000172+0000", 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000172+0000", 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000172+0000", 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000176+0000", 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000185+0000", 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.000191+0000", 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8000000000000004e-06 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.009428+0000", 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092370690000000005 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.009596+0000", 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718200000000001 2024-09-16T14:07:30.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.010079+0000", 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048326100000000002 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.011893+0000", 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018137299999999999 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.012329+0000", 2024-09-16T14:07:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000436239 2024-09-16T14:07:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.012647+0000", 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031829999999999998 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.012736+0000", 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9210999999999997e-05 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.269 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1818 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.046939+0000", 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.17174998, 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014369856, 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1818 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046939+0000", 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046939+0000", 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046940+0000", 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046945+0000", 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5339999999999999e-06 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046964+0000", 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8972000000000001e-05 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046973+0000", 2024-09-16T14:07:30.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.054337+0000", 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073648450000000001 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.054501+0000", 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016394099999999999 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.055062+0000", 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056039299999999996 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057928+0000", 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028667160000000001 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061233+0000", 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033044250000000002 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061276+0000", 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2793999999999997e-05 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061309+0000", 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3253000000000002e-05 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1819 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.046981+0000", 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.171708345000001, 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014748755000000001, 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1819 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046981+0000", 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046981+0000", 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046981+0000", 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8599999999999997e-07 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046986+0000", 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8239999999999999e-06 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.046997+0000", 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.047e-05 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047004+0000", 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8759999999999999e-06 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.055780+0000", 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0087759699999999993 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.055880+0000", 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9982000000000006e-05 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.056480+0000", 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00060014900000000002 2024-09-16T14:07:30.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061135+0000", 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046551320000000002 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061429+0000", 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029431900000000003 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061614+0000", 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018478599999999999 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061729+0000", 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115461 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1820 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.047012+0000", 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.171676973, 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014964712999999999, 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1820 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047012+0000", 2024-09-16T14:07:30.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047012+0000", 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047013+0000", 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9299999999999997e-07 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047017+0000", 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4100000000000001e-06 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047037+0000", 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9712e-05 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047054+0000", 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7099999999999999e-05 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.057980+0000", 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010925727999999999 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.058088+0000", 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010812800000000001 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.058703+0000", 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00061499700000000005 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061757+0000", 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030545260000000001 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061846+0000", 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8480000000000007e-05 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061918+0000", 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2336000000000007e-05 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.061977+0000", 2024-09-16T14:07:30.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8603000000000001e-05 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1821 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.047069+0000", 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.171619711, 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016629976000000001, 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1821 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047069+0000", 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047069+0000", 2024-09-16T14:07:30.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047070+0000", 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047075+0000", 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2820000000000004e-06 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047091+0000", 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.509e-05 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.047097+0000", 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.059396+0000", 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012298953 2024-09-16T14:07:30.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.059645+0000", 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000248409 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.060393+0000", 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00074888199999999998 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.062007+0000", 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001613401 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.063631+0000", 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016242559999999999 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.063668+0000", 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7452e-05 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.063699+0000", 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0811e-05 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3899 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:20.464653+0000", 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.7540359520000006, 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012489808999999999, 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3899 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464653+0000", 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464653+0000", 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464654+0000", 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464657+0000", 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3359999999999999e-06 2024-09-16T14:07:30.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464662+0000", 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2480000000000004e-06 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464666+0000", 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.365e-06 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473008+0000", 2024-09-16T14:07:30.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083424339999999993 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473102+0000", 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4099999999999997e-05 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.473371+0000", 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026890699999999998 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.474455+0000", 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010837539999999999 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477086+0000", 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026313489999999998 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477121+0000", 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4785e-05 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477143+0000", 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1960999999999999e-05 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3900 3.17 3:eed5514e:::rbd_object_map.1136494ba971:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:20.464669+0000", 2024-09-16T14:07:30.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.7540198349999994, 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012841968, 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3900 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464669+0000", 2024-09-16T14:07:30.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464669+0000", 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464670+0000", 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464672+0000", 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6579999999999998e-06 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464677+0000", 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1749999999999996e-06 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.464681+0000", 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5109999999999999e-06 2024-09-16T14:07:30.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.474531+0000", 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098500870000000004 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.474778+0000", 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000247431 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.475878+0000", 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010996739999999999 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477364+0000", 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014854460000000001 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477458+0000", 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4901000000000005e-05 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477489+0000", 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0892000000000002e-05 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:20.477511+0000", 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1810999999999998e-05 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7542 3.18 3:1cd9c7f6:::rbd_data.11369af4c40c.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.804895+0000", 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4137940960000002, 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017019457000000002, 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7542 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.804895+0000", 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.804895+0000", 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.804896+0000", 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0890000000000001e-06 2024-09-16T14:07:30.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808057+0000", 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031613840000000002 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808074+0000", 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808080+0000", 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2889999999999997e-06 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808145+0000", 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5693999999999998e-05 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808408+0000", 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026258400000000001 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809480+0000", 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010722640000000001 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821494+0000", 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012013249 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821798+0000", 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000304701 2024-09-16T14:07:30.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821864+0000", 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6214e-05 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821914+0000", 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9917000000000003e-05 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7544 3.1f 3:f86efcdd:::rbd_data.11369af4c40c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.808092+0000", 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4105972380000003, 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014525727, 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7544 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808092+0000", 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808092+0000", 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808093+0000", 2024-09-16T14:07:30.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6799999999999999e-07 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808098+0000", 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.383e-06 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808105+0000", 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.883e-06 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808108+0000", 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9909999999999998e-06 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808162+0000", 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4151999999999999e-05 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808339+0000", 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017649699999999999 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808637+0000", 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029818899999999997 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809503+0000", 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086654600000000005 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822551+0000", 2024-09-16T14:07:30.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013047303 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822588+0000", 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7673000000000003e-05 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822618+0000", 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9241999999999999e-05 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:7540 3.10 e35/23 3:0c0a7e4b:::rbd_data.11369af4c40c.0000000000000002:head v 35'40, mlcod=35'37)", 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.808130+0000", 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4105588410000003, 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013418888, 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808130+0000", 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808130+0000", 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808131+0000", 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809736+0000", 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001604865 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809754+0000", 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8057999999999999e-05 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809772+0000", 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8266000000000001e-05 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809817+0000", 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5302000000000001e-05 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809839+0000", 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.213e-05 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821498+0000", 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011658577999999999 2024-09-16T14:07:30.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.821549+0000", 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1196999999999998e-05 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7549 3.1f 3:fc2cd15a:::rbd_data.11369af4c40c.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.808781+0000", 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4099081660000001, 2024-09-16T14:07:30.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016760067, 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7549 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808781+0000", 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808781+0000", 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808782+0000", 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1500000000000003e-07 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811642+0000", 2024-09-16T14:07:30.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028604630000000002 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811659+0000", 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7022000000000001e-05 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811667+0000", 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6599999999999995e-06 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811769+0000", 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010229399999999999 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.812261+0000", 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049149199999999995 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814009+0000", 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017488040000000001 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.823432+0000", 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094223780000000004 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825298+0000", 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018666539999999999 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825439+0000", 2024-09-16T14:07:30.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014035799999999999 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825541+0000", 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010212700000000001 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7550 3.0 3:069bb4b1:::rbd_data.11369af4c40c.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.811687+0000", 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4070016020000002, 2024-09-16T14:07:30.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013961497999999999, 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7550 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811687+0000", 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811687+0000", 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.811689+0000", 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7540000000000001e-06 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814429+0000", 2024-09-16T14:07:30.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00274011 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814448+0000", 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9156000000000001e-05 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814455+0000", 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0060000000000003e-06 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814492+0000", 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6952999999999997e-05 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814629+0000", 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136475 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.815396+0000", 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076686799999999998 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822693+0000", 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072970450000000003 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825554+0000", 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028615670000000002 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825595+0000", 2024-09-16T14:07:30.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1044999999999998e-05 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825649+0000", 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3519e-05 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9727 3.1a e39/23 3:5b31ba1a:::rbd_data.1136a719415e.000000000000000c:head v 39'121, mlcod=39'118)", 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.0020957429999999, 2024-09-16T14:07:30.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026296277, 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216595+0000", 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7570000000000001e-06 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227934+0000", 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011338568 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227959+0000", 2024-09-16T14:07:30.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5874e-05 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227969+0000", 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.397e-06 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.228059+0000", 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9798999999999999e-05 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.228131+0000", 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2268000000000007e-05 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.242760+0000", 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014629085999999999 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.242890+0000", 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012952800000000001 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9793 3.1a e39/23 3:5f64584f:::rbd_data.1136a719415e.0000000000000002:head v 39'122, mlcod=39'120)", 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.352838+0000", 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.865850606, 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018344519, 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.352838+0000", 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.352838+0000", 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.352840+0000", 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8959999999999999e-06 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.362606+0000", 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097660630000000002 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.362641+0000", 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4516e-05 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.362652+0000", 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1073999999999999e-05 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.362730+0000", 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8369999999999997e-05 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.362770+0000", 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9912e-05 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.371134+0000", 2024-09-16T14:07:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083640889999999999 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.371183+0000", 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8599e-05 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.344 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:30.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:30.347 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:30.338+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:30.347 INFO:teuthology.orchestra.run.smithi079.stdout:1043 writesame 0xd07f73a thru 0xd089852 data_size 0x3b(0xa119 bytes) 2024-09-16T14:07:30.347 INFO:teuthology.orchestra.run.smithi079.stdout:1045 write 0xbb43dc7 thru 0xbb4d8cc (0x9b06 bytes) 2024-09-16T14:07:30.347 INFO:teuthology.orchestra.run.smithi079.stdout:1046 trunc from 0xd089853 to 0x59dbac3 2024-09-16T14:07:30.347 INFO:teuthology.orchestra.run.smithi079.stdout:1047 trunc from 0x59dbac3 to 0x32f9041 2024-09-16T14:07:30.348 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.348 INFO:teuthology.orchestra.run.smithi079.stdout:1051 read 0x9acd3a thru 0x9ad91a (0xbe1 bytes) 2024-09-16T14:07:30.348 INFO:teuthology.orchestra.run.smithi079.stdout:1053 write 0x7ec6709 thru 0x7ecf238 (0x8b30 bytes) 2024-09-16T14:07:30.348 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.348 INFO:teuthology.orchestra.run.smithi079.stdout:1056 write 0x7cd16b5 thru 0x7ce035e (0xecaa bytes) 2024-09-16T14:07:30.348 INFO:teuthology.orchestra.run.smithi079.stdout:1058 write 0xd57bd8b thru 0xd58058d (0x4803 bytes) 2024-09-16T14:07:30.349 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.349 INFO:teuthology.orchestra.run.smithi079.stdout:1061 write 0xb0a43ad thru 0xb0b2248 (0xde9c bytes) 2024-09-16T14:07:30.376 INFO:teuthology.orchestra.run.smithi079.stdout:1062 trunc from 0xd58058e to 0x2d7983b 2024-09-16T14:07:30.395 INFO:teuthology.orchestra.run.smithi079.stdout:1063 read 0x4b66ab thru 0x4c0931 (0xa287 bytes) 2024-09-16T14:07:30.397 INFO:teuthology.orchestra.run.smithi079.stdout:1064 trunc from 0x2d7983b to 0x1a4907f 2024-09-16T14:07:30.412 INFO:teuthology.orchestra.run.smithi079.stdout:1065 trunc from 0x1a4907f to 0xa78da6 2024-09-16T14:07:30.434 INFO:teuthology.orchestra.run.smithi079.stdout:1066 write 0x129b001 thru 0x129e2de (0x32de bytes) 2024-09-16T14:07:30.447 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:30.438+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:30.463 INFO:teuthology.orchestra.run.smithi079.stdout:1067 write 0x768affa thru 0x769994c (0xe953 bytes) 2024-09-16T14:07:30.469 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.469 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.469 INFO:teuthology.orchestra.run.smithi079.stdout:1070 read 0x4228f54 thru 0x42292fa (0x3a7 bytes) 2024-09-16T14:07:30.469 INFO:teuthology.orchestra.run.smithi079.stdout:1072 write 0x21a69b1 thru 0x21a924c (0x289c bytes) 2024-09-16T14:07:30.471 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.472 INFO:teuthology.orchestra.run.smithi079.stdout:1074 read 0x4f8aba9 thru 0x4f92012 (0x746a bytes) 2024-09-16T14:07:30.472 INFO:teuthology.orchestra.run.smithi079.stdout:1075 writesame 0x243f7c4 thru 0x244d539 data_size 0x21(0xdd76 bytes) 2024-09-16T14:07:30.474 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.474 INFO:teuthology.orchestra.run.smithi079.stdout:1079 read 0x4598da6 thru 0x459cae2 (0x3d3d bytes) 2024-09-16T14:07:30.474 INFO:teuthology.orchestra.run.smithi079.stdout:1080 read 0x5d49f47 thru 0x5d593ae (0xf468 bytes) 2024-09-16T14:07:30.475 INFO:teuthology.orchestra.run.smithi079.stdout:1081 read 0x1ebd49b thru 0x1ec47e8 (0x734e bytes) 2024-09-16T14:07:30.476 INFO:teuthology.orchestra.run.smithi079.stdout:1082 writesame 0x3719b26 thru 0x372298e data_size 0x1(0x8e69 bytes) 2024-09-16T14:07:30.509 INFO:teuthology.orchestra.run.smithi079.stdout:1083 read 0x5664fac thru 0x566cf99 (0x7fee bytes) 2024-09-16T14:07:30.509 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.509 INFO:teuthology.orchestra.run.smithi079.stdout:1087 read 0x12fa41a thru 0x130a27b (0xfe62 bytes) 2024-09-16T14:07:30.513 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:30.513 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:30.513 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:30.513 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:30.523 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:30.532 INFO:teuthology.orchestra.run.smithi079.stdout:1088 writesame 0xb7d4a7d thru 0xb7d7c6e data_size 0x3(0x31f2 bytes) 2024-09-16T14:07:30.532 INFO:teuthology.orchestra.run.smithi079.stdout:1091 write 0x505dc0a thru 0x5065b48 (0x7f3f bytes) 2024-09-16T14:07:30.538 INFO:teuthology.orchestra.run.smithi079.stdout:1092 write 0x4499991 thru 0x44a32d5 (0x9945 bytes) 2024-09-16T14:07:30.544 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.545 INFO:teuthology.orchestra.run.smithi079.stdout:1100 read 0x72bb06b thru 0x72c19d4 (0x696a bytes) 2024-09-16T14:07:30.545 INFO:teuthology.orchestra.run.smithi079.stdout:1102 read 0x4978192 thru 0x4986abd (0xe92c bytes) 2024-09-16T14:07:30.545 INFO:teuthology.orchestra.run.smithi079.stdout:1104 punch from 0x82debc8 to 0x82e0e10, (0x2248 bytes) 2024-09-16T14:07:30.545 INFO:teuthology.orchestra.run.smithi079.stdout:1106 punch from 0xd5f91f to 0xd606d0, (0xdb1 bytes) 2024-09-16T14:07:30.545 INFO:teuthology.orchestra.run.smithi079.stdout:1107 writesame 0x792a953 thru 0x7934fd9 data_size 0x1(0xa687 bytes) 2024-09-16T14:07:30.548 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:30.542+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:30.579 INFO:teuthology.orchestra.run.smithi079.stdout:1108 punch from 0x607e70a to 0x6088652, (0x9f48 bytes) 2024-09-16T14:07:30.579 INFO:teuthology.orchestra.run.smithi079.stdout:1109 punch from 0x2e1d28e to 0x2e2468a, (0x73fc bytes) 2024-09-16T14:07:30.582 INFO:teuthology.orchestra.run.smithi079.stdout:1110 trunc from 0xb7d7c6f to 0xa6f07ff 2024-09-16T14:07:30.595 INFO:teuthology.orchestra.run.smithi079.stdout:1111 punch from 0x399a682 to 0x39a40b1, (0x9a2f bytes) 2024-09-16T14:07:30.596 INFO:teuthology.orchestra.run.smithi079.stdout:1113 write 0x352cdac thru 0x35345f1 (0x7846 bytes) 2024-09-16T14:07:30.599 INFO:teuthology.orchestra.run.smithi079.stdout:1114 punch from 0x694fb17 to 0x6950b08, (0xff1 bytes) 2024-09-16T14:07:30.599 INFO:teuthology.orchestra.run.smithi079.stdout:1115 writesame 0x41b7ab1 thru 0x41c02c3 data_size 0x1(0x8813 bytes) 2024-09-16T14:07:30.620 INFO:teuthology.orchestra.run.smithi079.stdout:1116 writesame 0x7f7e116 thru 0x7f83fa8 data_size 0x1f(0x5e93 bytes) 2024-09-16T14:07:30.623 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:30.623 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:30.623 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:30.623 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:30.623 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:30.628 INFO:teuthology.orchestra.run.smithi079.stdout:1117 punch from 0x474622d to 0x474dab0, (0x7883 bytes) 2024-09-16T14:07:30.628 INFO:teuthology.orchestra.run.smithi079.stdout:1119 read 0x9d52eeb thru 0x9d57af6 (0x4c0c bytes) 2024-09-16T14:07:30.628 INFO:teuthology.orchestra.run.smithi079.stdout:1120 read 0xaa6f6d thru 0xaad431 (0x64c5 bytes) 2024-09-16T14:07:30.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:30.645 INFO:teuthology.orchestra.run.smithi079.stdout:1121 write 0xc27d7ca thru 0xc28b7b3 (0xdfea bytes) 2024-09-16T14:07:30.649 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:30.642+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:30.651 INFO:teuthology.orchestra.run.smithi079.stdout:1123 write 0xa02e012 thru 0xa03cf1c (0xef0b bytes) 2024-09-16T14:07:30.657 INFO:teuthology.orchestra.run.smithi079.stdout:1124 read 0xa031fef thru 0xa0399d5 (0x79e7 bytes) 2024-09-16T14:07:30.659 INFO:teuthology.orchestra.run.smithi079.stdout:1125 punch from 0x1f97c6f to 0x1fa4b97, (0xcf28 bytes) 2024-09-16T14:07:30.659 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.659 INFO:teuthology.orchestra.run.smithi079.stdout:1127 read 0xc3cfe9 thru 0xc4450a (0x7522 bytes) 2024-09-16T14:07:30.659 INFO:teuthology.orchestra.run.smithi079.stdout:1128 read 0xa5c2728 thru 0xa5c3c92 (0x156b bytes) 2024-09-16T14:07:30.660 INFO:teuthology.orchestra.run.smithi079.stdout:1129 write 0xa8a79c0 thru 0xa8ac118 (0x4759 bytes) 2024-09-16T14:07:30.667 INFO:teuthology.orchestra.run.smithi079.stdout:1130 writesame 0xad54579 thru 0xad583b8 data_size 0x40(0x3e40 bytes) 2024-09-16T14:07:30.670 INFO:teuthology.orchestra.run.smithi079.stdout:1131 write 0x53b788a thru 0x53c45fc (0xcd73 bytes) 2024-09-16T14:07:30.675 INFO:teuthology.orchestra.run.smithi079.stdout:1132 write 0x66bbce9 thru 0x66bbf22 (0x23a bytes) 2024-09-16T14:07:30.680 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:07:30.694 INFO:teuthology.orchestra.run.smithi079.stdout:1133 writesame 0xc766290 thru 0xc772dee data_size 0x1(0xcb5f bytes) 2024-09-16T14:07:30.695 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.695 INFO:teuthology.orchestra.run.smithi079.stdout:1135 writesame 0x56ecbd7 thru 0x56f86f9 data_size 0x3(0xbb23 bytes) 2024-09-16T14:07:30.709 INFO:teuthology.orchestra.run.smithi079.stdout:1136 read 0xae736f9 thru 0xae83292 (0xfb9a bytes) 2024-09-16T14:07:30.720 INFO:teuthology.orchestra.run.smithi079.stdout:1138 write 0xd0682d5 thru 0xd075a3f (0xd76b bytes) 2024-09-16T14:07:30.726 INFO:teuthology.orchestra.run.smithi079.stdout:1139 writesame 0x3c39cf4 thru 0x3c44d55 data_size 0x2(0xb062 bytes) 2024-09-16T14:07:30.731 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:30.731 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:30.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:30.731 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:30.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.731 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:884 3.12 3:4de31fa4:::rbd_data.1136e0d3367d.000000000000000a:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:30.732 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:30.732 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.014694917, 2024-09-16T14:07:30.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056685160000000002, 2024-09-16T14:07:30.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.733 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 884 2024-09-16T14:07:30.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.733 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:30.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711292+0000", 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711294+0000", 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7669999999999999e-06 2024-09-16T14:07:30.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711309+0000", 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4185999999999999e-05 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711391+0000", 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1997999999999997e-05 2024-09-16T14:07:30.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711408+0000", 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7782999999999999e-05 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711606+0000", 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019735900000000001 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.711909+0000", 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030280699999999999 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716960+0000", 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050516160000000001 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:894 3.1 3:834411d3:::rbd_data.1136e0d3367d.0000000000000011:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:30.738 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.01010468, 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067046559999999998, 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 894 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715882+0000", 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715885+0000", 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9299999999999999e-06 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715897+0000", 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2099999999999999e-05 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715943+0000", 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5785000000000003e-05 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.715970+0000", 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7529999999999999e-05 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716054+0000", 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4041000000000004e-05 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716135+0000", 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0524000000000004e-05 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.716338+0000", 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000203196 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.717189+0000", 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085096599999999998 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722487+0000", 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052979530000000002 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722541+0000", 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4268999999999998e-05 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.722587+0000", 2024-09-16T14:07:30.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5361999999999997e-05 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1551 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873068+0000", 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.852918127000001, 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053738780000000003, 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1551 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873068+0000", 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873068+0000", 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873069+0000", 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7999999999999993e-07 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873074+0000", 2024-09-16T14:07:30.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6630000000000001e-06 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873088+0000", 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3617e-05 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873096+0000", 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9570000000000002e-06 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878262+0000", 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005166187 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878348+0000", 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6055000000000002e-05 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878442+0000", 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4419000000000005e-05 2024-09-16T14:07:30.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1578 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873458+0000", 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.852528418, 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054174619999999996, 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1578 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873458+0000", 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873458+0000", 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873459+0000", 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.189e-06 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873466+0000", 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2020000000000001e-06 2024-09-16T14:07:30.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873475+0000", 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9340000000000001e-06 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873482+0000", 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4489999999999996e-06 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878722+0000", 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052398369999999998 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878791+0000", 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8814999999999998e-05 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878876+0000", 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5036000000000004e-05 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1579 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873535+0000", 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.852451204999999, 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055303089999999997, 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.755 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1579 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873535+0000", 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873535+0000", 2024-09-16T14:07:30.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873536+0000", 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.06e-06 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873540+0000", 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873550+0000", 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0811e-05 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873556+0000", 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3000000000000001e-06 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878915+0000", 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053588769999999997 2024-09-16T14:07:30.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.878983+0000", 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8497000000000005e-05 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879066+0000", 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2615999999999996e-05 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1580 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873561+0000", 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.852425588999999, 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005697967, 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1580 2024-09-16T14:07:30.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873561+0000", 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873561+0000", 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873562+0000", 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.09e-07 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873565+0000", 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9050000000000001e-06 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873572+0000", 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4200000000000001e-06 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873580+0000", 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4970000000000004e-06 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879104+0000", 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055246569999999997 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879172+0000", 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7815e-05 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879259+0000", 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6763999999999998e-05 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1581 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 352256~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873585+0000", 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.852401498000001, 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058658360000000001, 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1581 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873585+0000", 2024-09-16T14:07:30.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873585+0000", 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873586+0000", 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3399999999999999e-07 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873588+0000", 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4949999999999998e-06 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873595+0000", 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0280000000000003e-06 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873601+0000", 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879298+0000", 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056969430000000003 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879368+0000", 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0217999999999998e-05 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879451+0000", 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2800999999999995e-05 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1582 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873606+0000", 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.852380556, 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006036563, 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1582 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.771 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873606+0000", 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873606+0000", 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873607+0000", 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2300000000000001e-07 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873609+0000", 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5000000000000002e-06 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873616+0000", 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873621+0000", 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3170000000000003e-06 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879489+0000", 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058687239999999996 2024-09-16T14:07:30.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879559+0000", 2024-09-16T14:07:30.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9270000000000006e-05 2024-09-16T14:07:30.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879643+0000", 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4046000000000001e-05 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1583 3.1 3:81aa86d6:::rbd_data.11365ca8c0d.0000000000000011:6 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.873626+0000", 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.852361016, 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061920619999999999, 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1583 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873626+0000", 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873626+0000", 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873626+0000", 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2600000000000002e-07 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873629+0000", 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.225e-06 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873637+0000", 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2289999999999997e-06 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.873641+0000", 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3649999999999997e-06 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879667+0000", 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060265120000000004 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879736+0000", 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8127999999999996e-05 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.879818+0000", 2024-09-16T14:07:30.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1977000000000005e-05 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:1834 3.5 e31/23 3:a4605f64:::rbd_data.1136494ba971.00000000000001fa:head v 31'19, mlcod=31'15)", 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:18.077251+0000", 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.648735153000001, 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056664009999999997, 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077251+0000", 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077251+0000", 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077252+0000", 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.02e-06 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077261+0000", 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0650000000000005e-06 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077276+0000", 2024-09-16T14:07:30.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4871e-05 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077285+0000", 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4270000000000008e-06 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077360+0000", 2024-09-16T14:07:30.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4826999999999994e-05 2024-09-16T14:07:30.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.077380+0000", 2024-09-16T14:07:30.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.039e-05 2024-09-16T14:07:30.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.082847+0000", 2024-09-16T14:07:30.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054671470000000003 2024-09-16T14:07:30.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:18.082918+0000", 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0654000000000005e-05 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4128 3.1 3:80589f64:::rbd_data.1136494ba971.0000000000000033:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.435819+0000", 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.2901672019999992, 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059498670000000002, 2024-09-16T14:07:30.787 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.788 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4128 2024-09-16T14:07:30.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.788 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.435819+0000", 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.435819+0000", 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.435821+0000", 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7969999999999999e-06 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.435826+0000", 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0130000000000001e-06 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.435854+0000", 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8051e-05 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.435864+0000", 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0268999999999999e-05 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.435934+0000", 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9880000000000002e-05 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.436028+0000", 2024-09-16T14:07:30.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3189000000000005e-05 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.436162+0000", 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013444100000000001 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.437621+0000", 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001459393 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.441672+0000", 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040510729999999997 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.441730+0000", 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.441769+0000", 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9699000000000001e-05 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4150 3.7 3:e7112dea:::rbd_data.1136494ba971.0000000000000048:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.444674+0000", 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.2813125109999994, 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007632476, 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.794 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4150 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.444674+0000", 2024-09-16T14:07:30.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.444674+0000", 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.444675+0000", 2024-09-16T14:07:30.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T14:07:30.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.444682+0000", 2024-09-16T14:07:30.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5899999999999996e-06 2024-09-16T14:07:30.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.444697+0000", 2024-09-16T14:07:30.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5073e-05 2024-09-16T14:07:30.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.444705+0000", 2024-09-16T14:07:30.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6199999999999999e-06 2024-09-16T14:07:30.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.444915+0000", 2024-09-16T14:07:30.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021053200000000001 2024-09-16T14:07:30.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.444994+0000", 2024-09-16T14:07:30.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8819999999999994e-05 2024-09-16T14:07:30.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:30.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.445175+0000", 2024-09-16T14:07:30.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018095700000000001 2024-09-16T14:07:30.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.446545+0000", 2024-09-16T14:07:30.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001370362 2024-09-16T14:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.452231+0000", 2024-09-16T14:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056859040000000003 2024-09-16T14:07:30.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.452268+0000", 2024-09-16T14:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6355e-05 2024-09-16T14:07:30.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.452307+0000", 2024-09-16T14:07:30.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8875000000000003e-05 2024-09-16T14:07:30.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.806 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.806 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:7548 3.7 3:e0b211f1:::rbd_data.11369af4c40c.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:30.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.808318+0000", 2024-09-16T14:07:30.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.9176685129999997, 2024-09-16T14:07:30.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017691358000000001, 2024-09-16T14:07:30.807 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.807 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 7548 2024-09-16T14:07:30.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808318+0000", 2024-09-16T14:07:30.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808318+0000", 2024-09-16T14:07:30.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.808320+0000", 2024-09-16T14:07:30.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.883e-06 2024-09-16T14:07:30.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.815126+0000", 2024-09-16T14:07:30.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068062260000000003 2024-09-16T14:07:30.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.815145+0000", 2024-09-16T14:07:30.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8881000000000001e-05 2024-09-16T14:07:30.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.815154+0000", 2024-09-16T14:07:30.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2660000000000007e-06 2024-09-16T14:07:30.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.815242+0000", 2024-09-16T14:07:30.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7664999999999995e-05 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.815446+0000", 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020432100000000001 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.816225+0000", 2024-09-16T14:07:30.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077900500000000004 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.822368+0000", 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061425919999999997 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825930+0000", 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035620819999999998 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825976+0000", 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5822000000000003e-05 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.826009+0000", 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3615000000000003e-05 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.817 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:7542 3.18 e35/23 3:1cd9c7f6:::rbd_data.11369af4c40c.0000000000000004:head v 35'59, mlcod=35'56)", 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.809737+0000", 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.9162496679999999, 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007411323, 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.819 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809737+0000", 2024-09-16T14:07:30.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809737+0000", 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.809738+0000", 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4399999999999998e-07 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.810765+0000", 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010276650000000001 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.810788+0000", 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2120000000000002e-05 2024-09-16T14:07:30.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.810797+0000", 2024-09-16T14:07:30.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9970000000000004e-06 2024-09-16T14:07:30.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.810863+0000", 2024-09-16T14:07:30.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5908000000000007e-05 2024-09-16T14:07:30.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.810888+0000", 2024-09-16T14:07:30.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5933999999999999e-05 2024-09-16T14:07:30.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.817084+0000", 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061954180000000003 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.817148+0000", 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4337000000000004e-05 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.826 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:7539 3.15 e35/23 3:ae06ae40:::rbd_data.11369af4c40c.0000000000000001:head v 35'56, mlcod=35'53)", 2024-09-16T14:07:30.826 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.814500+0000", 2024-09-16T14:07:30.826 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.9114863480000004, 2024-09-16T14:07:30.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010974621, 2024-09-16T14:07:30.826 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.826 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814500+0000", 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814500+0000", 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.814502+0000", 2024-09-16T14:07:30.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.516e-06 2024-09-16T14:07:30.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.820856+0000", 2024-09-16T14:07:30.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063542249999999998 2024-09-16T14:07:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.820878+0000", 2024-09-16T14:07:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1605999999999999e-05 2024-09-16T14:07:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.820888+0000", 2024-09-16T14:07:30.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0121e-05 2024-09-16T14:07:30.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.820955+0000", 2024-09-16T14:07:30.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7478999999999995e-05 2024-09-16T14:07:30.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.820984+0000", 2024-09-16T14:07:30.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8436e-05 2024-09-16T14:07:30.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825426+0000", 2024-09-16T14:07:30.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044429040000000001 2024-09-16T14:07:30.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.825475+0000", 2024-09-16T14:07:30.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8334000000000002e-05 2024-09-16T14:07:30.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.832 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.833 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9779 3.7 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:30.833 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:30.833 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.432338777, 2024-09-16T14:07:30.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042761995999999997, 2024-09-16T14:07:30.833 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9779 2024-09-16T14:07:30.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:30.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:30.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293651+0000", 2024-09-16T14:07:30.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.021e-06 2024-09-16T14:07:30.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305709+0000", 2024-09-16T14:07:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012058627000000001 2024-09-16T14:07:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305754+0000", 2024-09-16T14:07:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4137000000000001e-05 2024-09-16T14:07:30.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305769+0000", 2024-09-16T14:07:30.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5257e-05 2024-09-16T14:07:30.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305971+0000", 2024-09-16T14:07:30.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000201833 2024-09-16T14:07:30.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.306248+0000", 2024-09-16T14:07:30.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027726999999999998 2024-09-16T14:07:30.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.307018+0000", 2024-09-16T14:07:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076971799999999996 2024-09-16T14:07:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.329087+0000", 2024-09-16T14:07:30.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022069787 2024-09-16T14:07:30.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336319+0000", 2024-09-16T14:07:30.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072311789999999999 2024-09-16T14:07:30.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336375+0000", 2024-09-16T14:07:30.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6406999999999997e-05 2024-09-16T14:07:30.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336410+0000", 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4759999999999999e-05 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9992 3.4 3:20df96d2:::rbd_data.1136a719415e.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.533911+0000", 2024-09-16T14:07:30.843 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1920752750000001, 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078353669999999993, 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9992 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.533911+0000", 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.533911+0000", 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.533913+0000", 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.153e-06 2024-09-16T14:07:30.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.533924+0000", 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.023e-05 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.533950+0000", 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5894e-05 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.533961+0000", 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766999999999999e-05 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.534058+0000", 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7021999999999994e-05 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.534370+0000", 2024-09-16T14:07:30.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031149500000000001 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.535954+0000", 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001583882 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.541560+0000", 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056059550000000001 2024-09-16T14:07:30.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.541581+0000", 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1413e-05 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.541679+0000", 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8003000000000003e-05 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.541747+0000", 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7552999999999995e-05 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:11930 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.041120+0000", 2024-09-16T14:07:30.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.68486625899999998, 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010318373, 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11930 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.041120+0000", 2024-09-16T14:07:30.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.041120+0000", 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.041122+0000", 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5e-06 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.043256+0000", 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002134014 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.043283+0000", 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7183999999999999e-05 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.043298+0000", 2024-09-16T14:07:30.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4921e-05 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.043398+0000", 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9889000000000005e-05 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.043636+0000", 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000238648 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.044040+0000", 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000403228 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.051033+0000", 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069928949999999998 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.051331+0000", 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029883399999999999 2024-09-16T14:07:30.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.051380+0000", 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8390999999999998e-05 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.051439+0000", 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8869000000000002e-05 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:11932 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:head [read 2850604~33753] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.044701+0000", 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.68128533599999996, 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071240820000000003, 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11932 2024-09-16T14:07:30.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.044701+0000", 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.044701+0000", 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.044703+0000", 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6130000000000001e-06 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.044709+0000", 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6949999999999997e-06 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.044724+0000", 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5119999999999999e-05 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.044731+0000", 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.255e-06 2024-09-16T14:07:30.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.050292+0000", 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055613629999999997 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for rw locks", 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.050425+0000", 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013294700000000001 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.051524+0000", 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001098985 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.051651+0000", 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012695600000000001 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.051825+0000", 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000174148 2024-09-16T14:07:30.862 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12172 3.12 3:4dae129f:::rbd_data.11364aaed44f.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.179043489, 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031007818999999999, 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:30.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12172 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:30.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546946+0000", 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2270000000000002e-06 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:30.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546958+0000", 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.173e-05 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546989+0000", 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1449000000000003e-05 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547001+0000", 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1216999999999999e-05 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547125+0000", 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012428299999999999 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547345+0000", 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022038799999999999 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.555417+0000", 2024-09-16T14:07:30.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080718119999999994 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577743+0000", 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02232599 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577765+0000", 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1675000000000002e-05 2024-09-16T14:07:30.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577879+0000", 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011386400000000001 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577951+0000", 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2183999999999999e-05 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.870 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:30.870 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:30.870 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:30.872 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:30.873 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:30.866+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1141 write 0xbab09fb thru 0xbab2aef (0x20f5 bytes) 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1144 write 0x32116a7 thru 0x322115b (0xfab5 bytes) 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1145 trunc from 0xd075a40 to 0x8be1fba 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1148 read 0xc4af08 thru 0xc582f9 (0xd3f2 bytes) 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1149 write 0xa01dfbf thru 0xa02a61c (0xc65e bytes) 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1151 read 0x3997bb7 thru 0x39a027c (0x86c6 bytes) 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1152 writesame 0x204a97c thru 0x2054d76 data_size 0x7(0xa3fb bytes) 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1153 write 0xb9013b8 thru 0xb90628f (0x4ed8 bytes) 2024-09-16T14:07:30.874 INFO:teuthology.orchestra.run.smithi079.stdout:1154 trunc from 0xb906290 to 0x8d4682a 2024-09-16T14:07:30.875 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.875 INFO:teuthology.orchestra.run.smithi079.stdout:1156 writesame 0x378b878 thru 0x3792d43 data_size 0x73(0x74cc bytes) 2024-09-16T14:07:30.875 INFO:teuthology.orchestra.run.smithi079.stdout:1159 trunc from 0x8d4682a to 0x8ccc6ea 2024-09-16T14:07:30.875 INFO:teuthology.orchestra.run.smithi079.stdout:1160 writesame 0x3890648 thru 0x3896d99 data_size 0x32(0x6752 bytes) 2024-09-16T14:07:30.875 INFO:teuthology.orchestra.run.smithi079.stdout:1161 punch from 0x29bb802 to 0x29c6162, (0xa960 bytes) 2024-09-16T14:07:30.875 INFO:teuthology.orchestra.run.smithi079.stdout:truncating to largest ever: 0xeb40c27 2024-09-16T14:07:30.884 INFO:teuthology.orchestra.run.smithi079.stdout:1162 trunc from 0x8ccc6ea to 0xeb40c27 2024-09-16T14:07:30.894 INFO:teuthology.orchestra.run.smithi079.stdout:1164 writesame 0xbb7fdda thru 0xbb87390 data_size 0x15(0x75b7 bytes) 2024-09-16T14:07:30.903 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.903 INFO:teuthology.orchestra.run.smithi079.stdout:1167 read 0xc27ecf2 thru 0xc28e091 (0xf3a0 bytes) 2024-09-16T14:07:30.904 INFO:teuthology.orchestra.run.smithi079.stdout:1169 read 0xa671a47 thru 0xa675a57 (0x4011 bytes) 2024-09-16T14:07:30.904 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.904 INFO:teuthology.orchestra.run.smithi079.stdout:1173 punch from 0x9eae7c6 to 0x9eb2465, (0x3c9f bytes) 2024-09-16T14:07:30.904 INFO:teuthology.orchestra.run.smithi079.stdout:1174 read 0x6919eff thru 0x6926d66 (0xce68 bytes) 2024-09-16T14:07:30.904 INFO:teuthology.orchestra.run.smithi079.stdout:1175 writesame 0x256b556 thru 0x2574a76 data_size 0x1(0x9521 bytes) 2024-09-16T14:07:30.936 INFO:teuthology.orchestra.run.smithi079.stdout:1176 read 0x50f623c thru 0x50fd034 (0x6df9 bytes) 2024-09-16T14:07:30.937 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:30.937 INFO:teuthology.orchestra.run.smithi079.stdout:1178 punch from 0x649296d to 0x64944f3, (0x1b86 bytes) 2024-09-16T14:07:30.937 INFO:teuthology.orchestra.run.smithi079.stdout:1180 writesame 0x4c554d4 thru 0x4c5cad2 data_size 0x1(0x75ff bytes) 2024-09-16T14:07:30.959 INFO:teuthology.orchestra.run.smithi079.stdout:1182 write 0x340835d thru 0x3408a3e (0x6e2 bytes) 2024-09-16T14:07:30.966 INFO:teuthology.orchestra.run.smithi079.stdout:1183 write 0xdffd0e1 thru 0xe00a0d1 (0xcff1 bytes) 2024-09-16T14:07:30.972 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:30.966+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:30.973 INFO:teuthology.orchestra.run.smithi079.stdout:1184 punch from 0xd469872 to 0xd46fec2, (0x6650 bytes) 2024-09-16T14:07:30.973 INFO:teuthology.orchestra.run.smithi079.stdout:1185 write 0x176dd92 thru 0x1779f33 (0xc1a2 bytes) 2024-09-16T14:07:30.976 INFO:teuthology.orchestra.run.smithi079.stdout:1186 writesame 0xbe0cad0 thru 0xbe1838a data_size 0x1(0xb8bb bytes) 2024-09-16T14:07:30.984 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:30.984 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:30.984 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:30.984 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:30.993 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:31.012 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.023 INFO:teuthology.orchestra.run.smithi079.stdout:1191 trunc from 0xeb40c27 to 0x8796c0b 2024-09-16T14:07:31.043 INFO:teuthology.orchestra.run.smithi079.stdout:1194 write 0x3bc19c4 thru 0x3bc1c22 (0x25f bytes) 2024-09-16T14:07:31.047 INFO:teuthology.orchestra.run.smithi079.stdout:1195 read 0xf38ace thru 0xf3e66b (0x5b9e bytes) 2024-09-16T14:07:31.048 INFO:teuthology.orchestra.run.smithi079.stdout:1196 trunc from 0x8796c0b to 0x8567478 2024-09-16T14:07:31.070 INFO:teuthology.orchestra.run.smithi079.stdout:1197 write 0x9a3d2cd thru 0x9a4656d (0x92a1 bytes) 2024-09-16T14:07:31.072 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:31.066+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:31.076 INFO:teuthology.orchestra.run.smithi079.stdout:1198 write 0x60ccebe thru 0x60d7315 (0xa458 bytes) 2024-09-16T14:07:31.092 INFO:teuthology.orchestra.run.smithi079.stdout:1202 write 0xa09ba83 thru 0xa0a770e (0xbc8c bytes) 2024-09-16T14:07:31.092 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:31.092 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:31.092 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:31.093 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:31.093 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:31.099 INFO:teuthology.orchestra.run.smithi079.stdout:1203 flatten 2024-09-16T14:07:31.102 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:31.112 INFO:teuthology.orchestra.run.smithi079.stdout:1204 read 0x2e419b thru 0x2f1726 (0xd58c bytes) 2024-09-16T14:07:31.114 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.127 INFO:teuthology.orchestra.run.smithi079.stdout:1206 writesame 0xafc8db4 thru 0xafd6882 data_size 0x11(0xdacf bytes) 2024-09-16T14:07:31.147 INFO:teuthology.orchestra.run.smithi079.stdout:1207 write 0xe3aada7 thru 0xe3abb60 (0xdba bytes) 2024-09-16T14:07:31.153 INFO:teuthology.orchestra.run.smithi079.stdout:1208 punch from 0x310ce09 to 0x31170cc, (0xa2c3 bytes) 2024-09-16T14:07:31.173 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:31.166+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:31.174 INFO:teuthology.orchestra.run.smithi079.stdout:1209 trunc from 0xe3abb61 to 0x39b3a72 2024-09-16T14:07:31.206 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:31.206 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:31.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:31.206 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:31.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.206 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:07:31.206 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:07:31.206 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.7921215190000002, 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:07:31.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:07:31.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:07:31.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:07:31.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4060 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 294912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.411735+0000", 2024-09-16T14:07:31.213 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7903228389999999, 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006175163, 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4060 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411735+0000", 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411735+0000", 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411736+0000", 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411739+0000", 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.977e-06 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411747+0000", 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2169999999999994e-06 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411752+0000", 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9309999999999999e-06 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.417677+0000", 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059248130000000001 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.417801+0000", 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012405700000000001 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.417910+0000", 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010945700000000001 2024-09-16T14:07:31.217 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4061 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 376832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.411759+0000", 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7902989260000002, 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063648480000000002, 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4061 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411759+0000", 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411759+0000", 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411760+0000", 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1999999999999999e-07 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411763+0000", 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2720000000000002e-06 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411771+0000", 2024-09-16T14:07:31.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.411775+0000", 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5859999999999998e-06 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.417947+0000", 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061722019999999999 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418035+0000", 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7501000000000001e-05 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418124+0000", 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8782000000000005e-05 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4065 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 98304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.222 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412374+0000", 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7896841329999997, 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062452569999999997, 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4065 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412374+0000", 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412374+0000", 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412375+0000", 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6799999999999999e-07 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412382+0000", 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9800000000000001e-06 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412390+0000", 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6260000000000001e-06 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412410+0000", 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0290999999999999e-05 2024-09-16T14:07:31.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418393+0000", 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059829879999999998 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418490+0000", 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6296000000000001e-05 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418619+0000", 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012920799999999999 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4066 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412417+0000", 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7896409169999998, 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063826170000000002, 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4066 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412417+0000", 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412417+0000", 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412418+0000", 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7400000000000002e-07 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412421+0000", 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0489999999999999e-06 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412429+0000", 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1620000000000008e-06 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412434+0000", 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9089999999999999e-06 2024-09-16T14:07:31.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418667+0000", 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062334399999999998 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418737+0000", 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9994999999999996e-05 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418800+0000", 2024-09-16T14:07:31.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2287999999999997e-05 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4067 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412554+0000", 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7895035910000008, 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064644630000000002, 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4067 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412554+0000", 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412554+0000", 2024-09-16T14:07:31.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412556+0000", 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.165e-06 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412561+0000", 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7400000000000004e-06 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412621+0000", 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9973000000000003e-05 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412644+0000", 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2442999999999998e-05 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418824+0000", 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061800140000000002 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418920+0000", 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6545999999999993e-05 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419019+0000", 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8581999999999999e-05 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4068 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 344064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412655+0000", 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7894032989999999, 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065592130000000004, 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4068 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412655+0000", 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412655+0000", 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412655+0000", 2024-09-16T14:07:31.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2799999999999995e-07 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412673+0000", 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.738e-05 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412697+0000", 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4634000000000002e-05 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412703+0000", 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0000000000000002e-06 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419059+0000", 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063559920000000004 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419136+0000", 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6946000000000005e-05 2024-09-16T14:07:31.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419214+0000", 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7433000000000002e-05 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4069 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 40960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412710+0000", 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7893474030000007, 2024-09-16T14:07:31.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006666763, 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4069 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412710+0000", 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412710+0000", 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412711+0000", 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1100000000000005e-07 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412714+0000", 2024-09-16T14:07:31.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1259999999999998e-06 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412723+0000", 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1939999999999998e-06 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412727+0000", 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5109999999999997e-06 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419256+0000", 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065286550000000004 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419317+0000", 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1629000000000002e-05 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419377+0000", 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9837000000000003e-05 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.246 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4070 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 122880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412732+0000", 2024-09-16T14:07:31.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7893254439999993, 2024-09-16T14:07:31.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068543459999999999, 2024-09-16T14:07:31.248 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.248 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4070 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412732+0000", 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412732+0000", 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412733+0000", 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412736+0000", 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6139999999999999e-06 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412744+0000", 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8569999999999995e-06 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412748+0000", 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6349999999999997e-06 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419406+0000", 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066574879999999996 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419490+0000", 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4263000000000003e-05 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419587+0000", 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6809000000000001e-05 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4071 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412762+0000", 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7892957529999993, 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070357520000000002, 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4071 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412762+0000", 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412762+0000", 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412763+0000", 2024-09-16T14:07:31.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7100000000000001e-07 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412774+0000", 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0669000000000001e-05 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412783+0000", 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7769999999999997e-06 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412789+0000", 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2820000000000004e-06 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419614+0000", 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068253159999999997 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419733+0000", 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119171 2024-09-16T14:07:31.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419798+0000", 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4765999999999996e-05 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4072 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412794+0000", 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7892635170000002, 2024-09-16T14:07:31.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072234769999999998, 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4072 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412794+0000", 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412794+0000", 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412795+0000", 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5799999999999998e-07 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412797+0000", 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2910000000000002e-06 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412805+0000", 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7260000000000002e-06 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412809+0000", 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3250000000000001e-06 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419822+0000", 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070127820000000004 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419932+0000", 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010974199999999999 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420018+0000", 2024-09-16T14:07:31.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5853000000000006e-05 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4073 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412815+0000", 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7892431230000003, 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074833160000000003, 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4073 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412815+0000", 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412815+0000", 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412815+0000", 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.37e-07 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412818+0000", 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3199999999999998e-06 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412825+0000", 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4800000000000004e-06 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412830+0000", 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6e-06 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420064+0000", 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072340640000000001 2024-09-16T14:07:31.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420193+0000", 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012862299999999999 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420298+0000", 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105492 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4074 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 57344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413225+0000", 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.788832932, 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073554960000000004, 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4074 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.267 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413225+0000", 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413225+0000", 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413227+0000", 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413233+0000", 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6830000000000002e-06 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413278+0000", 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4919000000000001e-05 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413297+0000", 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8244999999999999e-05 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420357+0000", 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070600669999999997 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420472+0000", 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011545899999999999 2024-09-16T14:07:31.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420580+0000", 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108263 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4075 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 139264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413305+0000", 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7887523729999995, 2024-09-16T14:07:31.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075430260000000004, 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4075 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413305+0000", 2024-09-16T14:07:31.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413305+0000", 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413307+0000", 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0249999999999999e-06 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413310+0000", 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.783e-06 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413319+0000", 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5799999999999992e-06 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413323+0000", 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420624+0000", 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073002049999999997 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420716+0000", 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2466999999999994e-05 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420849+0000", 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013234 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4076 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413328+0000", 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7887294740000002, 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077674240000000002, 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4076 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413328+0000", 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413328+0000", 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413329+0000", 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.54e-07 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413332+0000", 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T14:07:31.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413341+0000", 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5659999999999996e-06 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413346+0000", 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0649999999999998e-06 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420890+0000", 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007544083 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420999+0000", 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010828300000000001 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421096+0000", 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7070000000000004e-05 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.280 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4077 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 303104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413352+0000", 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7887062969999992, 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079281709999999995, 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.281 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4077 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413352+0000", 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413352+0000", 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413352+0000", 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.54e-07 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413355+0000", 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3690000000000001e-06 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413364+0000", 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4110000000000002e-06 2024-09-16T14:07:31.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413369+0000", 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421121+0000", 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077525240000000002 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421211+0000", 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9825000000000001e-05 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421280+0000", 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8563000000000001e-05 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4078 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.285 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413374+0000", 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7886834890000003, 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080617139999999993, 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4078 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413374+0000", 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413374+0000", 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413375+0000", 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413377+0000", 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2400000000000002e-06 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413385+0000", 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8890000000000002e-06 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413390+0000", 2024-09-16T14:07:31.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6970000000000001e-06 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421310+0000", 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079205270000000001 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421388+0000", 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7106000000000003e-05 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421436+0000", 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8544000000000001e-05 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9028 3.b 3:d1d3b7e8:::rbd_data.1136dd12cea0.0000000000000023:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:25.817264+0000", 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3847934080000002, 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010794115999999999, 2024-09-16T14:07:31.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9028 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817264+0000", 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817264+0000", 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817267+0000", 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817281+0000", 2024-09-16T14:07:31.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4104000000000001e-05 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817329+0000", 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7669e-05 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817350+0000", 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0735000000000001e-05 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817519+0000", 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016912700000000001 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817529+0000", 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0266e-05 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.823533+0000", 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060039530000000002 2024-09-16T14:07:31.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.823660+0000", 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000126365 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.823987+0000", 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032733600000000001 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:31.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.827260+0000", 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032734090000000001 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.827984+0000", 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072412099999999999 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.828016+0000", 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1809e-05 2024-09-16T14:07:31.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.828059+0000", 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2286000000000001e-05 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9033 3.b 3:d480079a:::rbd_data.1136dd12cea0.0000000000000032:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:25.822388+0000", 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.379669399, 2024-09-16T14:07:31.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069682609999999999, 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9033 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822388+0000", 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822388+0000", 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822391+0000", 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6299999999999998e-06 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822404+0000", 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3135999999999999e-05 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822449+0000", 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4489e-05 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822468+0000", 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8899e-05 2024-09-16T14:07:31.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822684+0000", 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000216293 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822696+0000", 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1610000000000001e-05 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.825986+0000", 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032903450000000001 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.826092+0000", 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106159 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.826386+0000", 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029419899999999998 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.827595+0000", 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001208403 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.829256+0000", 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016611169999999999 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.829307+0000", 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0803000000000001e-05 2024-09-16T14:07:31.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.829357+0000", 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0178e-05 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9364 3.b 3:d0c4c146:::rbd_id.image_client.0-clone7:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:26.800242+0000", 2024-09-16T14:07:31.304 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4018160970000002, 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062682140000000003, 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9364 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:31.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.800242+0000", 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.800242+0000", 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.800244+0000", 2024-09-16T14:07:31.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7439999999999999e-06 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.800257+0000", 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2437e-05 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.800293+0000", 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6254999999999998e-05 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.800305+0000", 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2157000000000001e-05 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.800404+0000", 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8715000000000006e-05 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.800412+0000", 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3629999999999994e-06 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.803730+0000", 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033176780000000001 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.803910+0000", 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000180013 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.804124+0000", 2024-09-16T14:07:31.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021407699999999999 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.805206+0000", 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010813820000000001 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.806405+0000", 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001199007 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.806467+0000", 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1951000000000004e-05 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:26.806510+0000", 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3435e-05 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:31.311 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:31.314 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:31.306+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:31.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:31.315 INFO:teuthology.orchestra.run.smithi079.stdout:1211 write 0xd05f6e3 thru 0xd06acbd (0xb5db bytes) 2024-09-16T14:07:31.315 INFO:teuthology.orchestra.run.smithi079.stdout:1213 read 0xbfb5d30 thru 0xbfb9701 (0x39d2 bytes) 2024-09-16T14:07:31.315 INFO:teuthology.orchestra.run.smithi079.stdout:1214 punch from 0x956bf09 to 0x9576a96, (0xab8d bytes) 2024-09-16T14:07:31.315 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.315 INFO:teuthology.orchestra.run.smithi079.stdout:1216 read 0x2ce3d32 thru 0x2ceaaf1 (0x6dc0 bytes) 2024-09-16T14:07:31.315 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:1218 read 0x1fe78fd thru 0x1feb937 (0x403b bytes) 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:1223 write 0xa60987d thru 0xa60e678 (0x4dfc bytes) 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:1224 trunc from 0xd06acbe to 0x3a9d55c 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:1225 punch from 0x337ffdb to 0x338f1d8, (0xf1fd bytes) 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:1226 trunc from 0x3a9d55c to 0x5aceb51 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:1227 read 0x31b36a9 thru 0x31b8b10 (0x5468 bytes) 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:1229 writesame 0xc4d2083 thru 0xc4d7e2f data_size 0x1(0x5dad bytes) 2024-09-16T14:07:31.316 INFO:teuthology.orchestra.run.smithi079.stdout:1230 writesame 0x1d09344 thru 0x1d0d4c1 data_size 0x2(0x417e bytes) 2024-09-16T14:07:31.343 INFO:teuthology.orchestra.run.smithi079.stdout:1231 trunc from 0xc4d7e30 to 0x190de0f 2024-09-16T14:07:31.363 INFO:teuthology.orchestra.run.smithi079.stdout:1232 read 0x9879a thru 0xa6779 (0xdfe0 bytes) 2024-09-16T14:07:31.364 INFO:teuthology.orchestra.run.smithi079.stdout:1233 punch from 0x97eee6 to 0x9812d8, (0x23f2 bytes) 2024-09-16T14:07:31.381 INFO:teuthology.orchestra.run.smithi079.stdout:1234 write 0x78430ea thru 0x784e962 (0xb879 bytes) 2024-09-16T14:07:31.388 INFO:teuthology.orchestra.run.smithi079.stdout:1235 trunc from 0x784e963 to 0x6d4ee6a 2024-09-16T14:07:31.414 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:31.406+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:31.428 INFO:teuthology.orchestra.run.smithi079.stdout:1236 writesame 0xde5de5b thru 0xde66d68 data_size 0x1(0x8f0e bytes) 2024-09-16T14:07:31.428 INFO:teuthology.orchestra.run.smithi079.stdout:1237 write 0xdb919a thru 0xdbb921 (0x2788 bytes) 2024-09-16T14:07:31.429 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:31.429 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:31.429 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:31.429 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:31.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:31.447 INFO:teuthology.orchestra.run.smithi079.stdout:1238 trunc from 0xde66d69 to 0x2b2a627 2024-09-16T14:07:31.457 INFO:teuthology.orchestra.run.smithi079.stdout:1239 trunc from 0x2b2a627 to 0x36541cf 2024-09-16T14:07:31.465 INFO:teuthology.orchestra.run.smithi079.stdout:1241 punch from 0x7b02b9 to 0x7b68d0, (0x6617 bytes) 2024-09-16T14:07:31.465 INFO:teuthology.orchestra.run.smithi079.stdout:1242 punch from 0x29b7251 to 0x29b9811, (0x25c0 bytes) 2024-09-16T14:07:31.465 INFO:teuthology.orchestra.run.smithi079.stdout:1243 writesame 0x29ec564 thru 0x29fbaba data_size 0x1(0xf557 bytes) 2024-09-16T14:07:31.485 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:07:31.485 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:07:31.485 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T14:07:31.485 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T14:07:31.485 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T14:07:31.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-16T14:07:31.514 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:31.506+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:31.533 INFO:teuthology.orchestra.run.smithi079.stdout:1244 writesame 0xb13936b thru 0xb148b26 data_size 0x8c(0xf7bc bytes) 2024-09-16T14:07:31.533 INFO:teuthology.orchestra.run.smithi079.stdout:1245 read 0x62899af thru 0x62906c4 (0x6d16 bytes) 2024-09-16T14:07:31.533 INFO:teuthology.orchestra.run.smithi079.stdout:1246 write 0x5059fef thru 0x50666ee (0xc700 bytes) 2024-09-16T14:07:31.541 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:31.542 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:31.542 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:31.542 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:31.542 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:31.549 INFO:teuthology.orchestra.run.smithi079.stdout:1247 write 0xcf883b2 thru 0xcf96c73 (0xe8c2 bytes) 2024-09-16T14:07:31.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:31.554 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.555 INFO:teuthology.orchestra.run.smithi079.stdout:1250 trunc from 0xcf96c74 to 0xccf1f8c 2024-09-16T14:07:31.567 INFO:teuthology.orchestra.run.smithi079.stdout:1251 trunc from 0xccf1f8c to 0xe8a5795 2024-09-16T14:07:31.575 INFO:teuthology.orchestra.run.smithi079.stdout:1253 read 0x6b0328c thru 0x6b0dae7 (0xa85c bytes) 2024-09-16T14:07:31.575 INFO:teuthology.orchestra.run.smithi079.stdout:1256 read 0x36d9b3a thru 0x36de37b (0x4842 bytes) 2024-09-16T14:07:31.575 INFO:teuthology.orchestra.run.smithi079.stdout:1258 write 0x1847d48 thru 0x184d216 (0x54cf bytes) 2024-09-16T14:07:31.594 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T14:07:31.594 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:31.598 INFO:teuthology.orchestra.run.smithi079.stdout:1259 trunc from 0xe8a5795 to 0x3b8b93 2024-09-16T14:07:31.615 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:31.610+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:31.638 INFO:teuthology.orchestra.run.smithi079.stdout:1261 writesame 0x7fa2ac8 thru 0x7fa69d2 data_size 0x1(0x3f0b bytes) 2024-09-16T14:07:31.638 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.638 INFO:teuthology.orchestra.run.smithi079.stdout:1263 read 0x6945cd8 thru 0x694d168 (0x7491 bytes) 2024-09-16T14:07:31.648 INFO:teuthology.orchestra.run.smithi079.stdout:1264 trunc from 0x7fa69d3 to 0x14dfe33 2024-09-16T14:07:31.662 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4850 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.711267+0000", 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9484810699999997, 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014202039, 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4850 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711267+0000", 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711267+0000", 2024-09-16T14:07:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711268+0000", 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8599999999999999e-07 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711269+0000", 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8449999999999999e-06 2024-09-16T14:07:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711277+0000", 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2570000000000001e-06 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711281+0000", 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.722855+0000", 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011573685 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.722929+0000", 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399000000000004e-05 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725469+0000", 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002540297 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4851 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.711286+0000", 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9484627519999993, 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014377928, 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4851 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711286+0000", 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711286+0000", 2024-09-16T14:07:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711286+0000", 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711293+0000", 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8299999999999998e-06 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711302+0000", 2024-09-16T14:07:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2860000000000005e-06 2024-09-16T14:07:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711307+0000", 2024-09-16T14:07:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7389999999999999e-06 2024-09-16T14:07:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725508+0000", 2024-09-16T14:07:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014201266000000001 2024-09-16T14:07:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725588+0000", 2024-09-16T14:07:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0080999999999994e-05 2024-09-16T14:07:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725664+0000", 2024-09-16T14:07:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5343e-05 2024-09-16T14:07:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5053 3.1b 3:dd61974c:::rbd_data.1136494ba971.0000000000000074:7 [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715261+0000", 2024-09-16T14:07:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9444873929999993, 2024-09-16T14:07:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01377916, 2024-09-16T14:07:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5053 2024-09-16T14:07:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715261+0000", 2024-09-16T14:07:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715261+0000", 2024-09-16T14:07:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715261+0000", 2024-09-16T14:07:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-16T14:07:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715263+0000", 2024-09-16T14:07:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:07:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715269+0000", 2024-09-16T14:07:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9059999999999999e-06 2024-09-16T14:07:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715273+0000", 2024-09-16T14:07:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2379999999999999e-06 2024-09-16T14:07:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.728884+0000", 2024-09-16T14:07:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013610344 2024-09-16T14:07:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.728970+0000", 2024-09-16T14:07:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6846000000000002e-05 2024-09-16T14:07:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729040+0000", 2024-09-16T14:07:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9599999999999998e-05 2024-09-16T14:07:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5054 3.1b 3:dd61974c:::rbd_data.1136494ba971.0000000000000074:7 [read 196608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715278+0000", 2024-09-16T14:07:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9444707730000008, 2024-09-16T14:07:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013927065000000001, 2024-09-16T14:07:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5054 2024-09-16T14:07:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715278+0000", 2024-09-16T14:07:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715278+0000", 2024-09-16T14:07:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715278+0000", 2024-09-16T14:07:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4700000000000001e-07 2024-09-16T14:07:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715280+0000", 2024-09-16T14:07:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.024e-06 2024-09-16T14:07:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715286+0000", 2024-09-16T14:07:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2910000000000004e-06 2024-09-16T14:07:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715290+0000", 2024-09-16T14:07:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:07:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729066+0000", 2024-09-16T14:07:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013776149999999999 2024-09-16T14:07:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729139+0000", 2024-09-16T14:07:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2812999999999996e-05 2024-09-16T14:07:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729205+0000", 2024-09-16T14:07:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5314999999999994e-05 2024-09-16T14:07:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5062 3.1b 3:dfaae061:::rbd_data.1136494ba971.000000000000007a:7 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715296+0000", 2024-09-16T14:07:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9444525549999998, 2024-09-16T14:07:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014091349, 2024-09-16T14:07:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5062 2024-09-16T14:07:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715296+0000", 2024-09-16T14:07:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715296+0000", 2024-09-16T14:07:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715296+0000", 2024-09-16T14:07:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:07:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715298+0000", 2024-09-16T14:07:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1919999999999999e-06 2024-09-16T14:07:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715305+0000", 2024-09-16T14:07:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6880000000000002e-06 2024-09-16T14:07:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715309+0000", 2024-09-16T14:07:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4470000000000002e-06 2024-09-16T14:07:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729229+0000", 2024-09-16T14:07:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013920069 2024-09-16T14:07:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729313+0000", 2024-09-16T14:07:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4788000000000002e-05 2024-09-16T14:07:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729387+0000", 2024-09-16T14:07:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3736e-05 2024-09-16T14:07:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5063 3.1b 3:dfaae061:::rbd_data.1136494ba971.000000000000007a:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715313+0000", 2024-09-16T14:07:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9444351550000007, 2024-09-16T14:07:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014351422000000001, 2024-09-16T14:07:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.709 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5063 2024-09-16T14:07:31.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.709 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715313+0000", 2024-09-16T14:07:31.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715313+0000", 2024-09-16T14:07:31.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715314+0000", 2024-09-16T14:07:31.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9800000000000003e-07 2024-09-16T14:07:31.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715316+0000", 2024-09-16T14:07:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0540000000000002e-06 2024-09-16T14:07:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715322+0000", 2024-09-16T14:07:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5320000000000003e-06 2024-09-16T14:07:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715326+0000", 2024-09-16T14:07:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7720000000000001e-06 2024-09-16T14:07:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729523+0000", 2024-09-16T14:07:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014197052 2024-09-16T14:07:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729607+0000", 2024-09-16T14:07:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3993999999999996e-05 2024-09-16T14:07:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729665+0000", 2024-09-16T14:07:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.732e-05 2024-09-16T14:07:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5085 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 16384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9444182869999995, 2024-09-16T14:07:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015613336, 2024-09-16T14:07:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5085 2024-09-16T14:07:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0200000000000003e-07 2024-09-16T14:07:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715333+0000", 2024-09-16T14:07:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:07:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715339+0000", 2024-09-16T14:07:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3629999999999999e-06 2024-09-16T14:07:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715343+0000", 2024-09-16T14:07:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.924e-06 2024-09-16T14:07:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729409+0000", 2024-09-16T14:07:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014065846999999999 2024-09-16T14:07:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.730871+0000", 2024-09-16T14:07:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014620119999999999 2024-09-16T14:07:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.730943+0000", 2024-09-16T14:07:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2521000000000006e-05 2024-09-16T14:07:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5086 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 98304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9444015080000003, 2024-09-16T14:07:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015853791999999998, 2024-09-16T14:07:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5086 2024-09-16T14:07:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3799999999999997e-07 2024-09-16T14:07:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715349+0000", 2024-09-16T14:07:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.136e-06 2024-09-16T14:07:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715356+0000", 2024-09-16T14:07:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1290000000000001e-06 2024-09-16T14:07:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715359+0000", 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.495e-06 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731021+0000", 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015662367999999999 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731115+0000", 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3377999999999999e-05 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731201+0000", 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5748000000000006e-05 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5087 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9443851119999991, 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016037577000000001, 2024-09-16T14:07:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5087 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715364+0000", 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5100000000000001e-07 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715366+0000", 2024-09-16T14:07:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715371+0000", 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3469999999999998e-06 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715374+0000", 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4450000000000001e-06 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731249+0000", 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015874528999999998 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731332+0000", 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731401+0000", 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9209999999999996e-05 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5097 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9443198580000001, 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016148116000000001, 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5097 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715429+0000", 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0500000000000004e-07 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715432+0000", 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9569999999999998e-06 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715438+0000", 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2519999999999996e-06 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715442+0000", 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8130000000000002e-06 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731427+0000", 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015984851000000001 2024-09-16T14:07:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731498+0000", 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1260999999999994e-05 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731577+0000", 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8477000000000001e-05 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5098 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9443024139999991, 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016302106, 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5098 2024-09-16T14:07:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715448+0000", 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.852e-06 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715455+0000", 2024-09-16T14:07:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3720000000000001e-06 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715459+0000", 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0990000000000001e-06 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731602+0000", 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016142897 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731668+0000", 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6161999999999994e-05 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731748+0000", 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0314000000000004e-05 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5311 3.1b 3:dfaae061:::rbd_data.1136494ba971.000000000000007a:7 [read 196608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723510+0000", 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9362386770000004, 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014688708999999999, 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5311 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723510+0000", 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723510+0000", 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723510+0000", 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7899999999999999e-07 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723512+0000", 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7430000000000001e-06 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723518+0000", 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7799999999999997e-06 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723522+0000", 2024-09-16T14:07:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5179999999999998e-06 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738050+0000", 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014528078999999999 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738128+0000", 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8140999999999996e-05 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738198+0000", 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0068999999999996e-05 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5349 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723619+0000", 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9361288630000004, 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013772769000000001, 2024-09-16T14:07:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5349 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723619+0000", 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723619+0000", 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723620+0000", 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723621+0000", 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6470000000000001e-06 2024-09-16T14:07:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723627+0000", 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.75e-06 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723631+0000", 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737228+0000", 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013596949000000001 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737305+0000", 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6516999999999999e-05 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737392+0000", 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7682999999999994e-05 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5350 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723635+0000", 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9361132879999996, 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013958325000000001, 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5350 2024-09-16T14:07:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723635+0000", 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723635+0000", 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723636+0000", 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5899999999999996e-07 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723640+0000", 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3789999999999999e-06 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723646+0000", 2024-09-16T14:07:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8919999999999997e-06 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723651+0000", 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2240000000000003e-06 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737414+0000", 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013763266 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737510+0000", 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5483999999999996e-05 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737593+0000", 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3421e-05 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5421 3.3 3:c5757882:::rbd_data.1136494ba971.000000000000002c:7 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723848+0000", 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9359006789999995, 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013924826, 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5421 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723848+0000", 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723848+0000", 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723848+0000", 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:07:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723850+0000", 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6959999999999999e-06 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723856+0000", 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1059999999999996e-06 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723860+0000", 2024-09-16T14:07:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737616+0000", 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013756286 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737691+0000", 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5277000000000004e-05 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737772+0000", 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1213999999999999e-05 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5422 3.3 3:c5757882:::rbd_data.1136494ba971.000000000000002c:7 [read 81920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723864+0000", 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9358847650000008, 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014087064, 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5422 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723864+0000", 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723864+0000", 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723864+0000", 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723866+0000", 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.846e-06 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723872+0000", 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1299999999999998e-06 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723876+0000", 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7100000000000001e-06 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737794+0000", 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013918363 2024-09-16T14:07:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737885+0000", 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0586000000000003e-05 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737951+0000", 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6021999999999999e-05 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5423 3.3 3:c5757882:::rbd_data.1136494ba971.000000000000002c:7 [read 172032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723882+0000", 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9358662760000005, 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014510959, 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5423 2024-09-16T14:07:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723882+0000", 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723882+0000", 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723882+0000", 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723884+0000", 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.652e-06 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723891+0000", 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6830000000000002e-06 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723895+0000", 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737973+0000", 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014078146999999999 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738279+0000", 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030630399999999997 2024-09-16T14:07:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738393+0000", 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113818 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5424 3.3 3:c5757882:::rbd_data.1136494ba971.000000000000002c:7 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723899+0000", 2024-09-16T14:07:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9358496439999993, 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015007779000000001, 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5424 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723899+0000", 2024-09-16T14:07:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723899+0000", 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723899+0000", 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723901+0000", 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8700000000000001e-06 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723907+0000", 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0179999999999998e-06 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723911+0000", 2024-09-16T14:07:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8909999999999997e-06 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738581+0000", 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014670337 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738656+0000", 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4598000000000006e-05 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738906+0000", 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025068099999999999 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5461 3.1b 3:d9580610:::rbd_data.1136494ba971.000000000000003a:7 [read 73728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723915+0000", 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9358336359999999, 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014811758, 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5461 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723915+0000", 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723915+0000", 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723915+0000", 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723922+0000", 2024-09-16T14:07:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9879999999999998e-06 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723929+0000", 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4570000000000002e-06 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723932+0000", 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5939999999999998e-06 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738418+0000", 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014485736000000001 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738495+0000", 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6792999999999995e-05 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738726+0000", 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023179400000000001 2024-09-16T14:07:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5462 3.1b 3:d9580610:::rbd_data.1136494ba971.000000000000003a:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723936+0000", 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9358118879999999, 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014951133, 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5462 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723936+0000", 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723936+0000", 2024-09-16T14:07:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723937+0000", 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.03e-07 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723939+0000", 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9300000000000002e-06 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723945+0000", 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0560000000000001e-06 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723949+0000", 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7409999999999998e-06 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738748+0000", 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014799909999999999 2024-09-16T14:07:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738822+0000", 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3542999999999997e-05 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738888+0000", 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5549999999999994e-05 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:31.792 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:31.793 INFO:teuthology.orchestra.run.smithi079.stdout:1266 trunc from 0x14dfe33 to 0x66da735 2024-09-16T14:07:31.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:31.795 INFO:teuthology.orchestra.run.smithi079.stdout:1267 write 0x6814d10 thru 0x681b7f3 (0x6ae4 bytes) 2024-09-16T14:07:31.795 INFO:teuthology.orchestra.run.smithi079.stdout:1268 read 0x2f2a56a thru 0x2f387d8 (0xe26f bytes) 2024-09-16T14:07:31.795 INFO:teuthology.orchestra.run.smithi079.stdout:1269 write 0xbd637fd thru 0xbd6c007 (0x880b bytes) 2024-09-16T14:07:31.795 INFO:teuthology.orchestra.run.smithi079.stdout:1270 read 0x405b9fd thru 0x4062d84 (0x7388 bytes) 2024-09-16T14:07:31.795 INFO:teuthology.orchestra.run.smithi079.stdout:1272 read 0x98c1128 thru 0x98c9455 (0x832e bytes) 2024-09-16T14:07:31.795 INFO:teuthology.orchestra.run.smithi079.stdout:1273 read 0x599c408 thru 0x59a9314 (0xcf0d bytes) 2024-09-16T14:07:31.796 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.796 INFO:teuthology.orchestra.run.smithi079.stdout:1275 trunc from 0xbd6c008 to 0xde547bf 2024-09-16T14:07:31.796 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:31.796 INFO:teuthology.orchestra.run.smithi079.stdout:1277 write 0x11cca84 thru 0x11d2b45 (0x60c2 bytes) 2024-09-16T14:07:31.796 INFO:teuthology.orchestra.run.smithi079.stdout:1278 punch from 0x12c902d to 0x12d0b94, (0x7b67 bytes) 2024-09-16T14:07:31.796 INFO:teuthology.orchestra.run.smithi079.stdout:1279 clone 9 order 18 su 65536 sc 8 2024-09-16T14:07:31.796 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:31.786+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:31.894 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:31.886+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:31.904 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:31.904 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:31.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:31.995 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:31.990+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:32.011 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:32.012 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:32.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:32.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:32.097 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:32.094+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.270837989999997, 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15070547300000001, 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847226+0000", 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.204e-06 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847235+0000", 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847250+0000", 2024-09-16T14:07:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5662e-05 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847261+0000", 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0236e-05 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847327+0000", 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.622e-05 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847345+0000", 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8434e-05 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.993706+0000", 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14636041899999999 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994117+0000", 2024-09-16T14:07:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041132499999999998 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994428+0000", 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031060899999999999 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.995845+0000", 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014173359999999999 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997768+0000", 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001922991 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997867+0000", 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9057999999999999e-05 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997929+0000", 2024-09-16T14:07:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2433000000000003e-05 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.891586357000001, 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013812964, 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 48 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226477+0000", 2024-09-16T14:07:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.879999999999999e-07 2024-09-16T14:07:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226482+0000", 2024-09-16T14:07:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0919999999999997e-06 2024-09-16T14:07:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226537+0000", 2024-09-16T14:07:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4916000000000001e-05 2024-09-16T14:07:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226546+0000", 2024-09-16T14:07:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9709999999999999e-06 2024-09-16T14:07:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236148+0000", 2024-09-16T14:07:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0096011849999999999 2024-09-16T14:07:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236208+0000", 2024-09-16T14:07:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9922e-05 2024-09-16T14:07:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236308+0000", 2024-09-16T14:07:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010068000000000001 2024-09-16T14:07:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239183+0000", 2024-09-16T14:07:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002874779 2024-09-16T14:07:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240208+0000", 2024-09-16T14:07:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001025312 2024-09-16T14:07:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240255+0000", 2024-09-16T14:07:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6363999999999999e-05 2024-09-16T14:07:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240288+0000", 2024-09-16T14:07:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3754999999999998e-05 2024-09-16T14:07:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.891507255999997, 2024-09-16T14:07:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013937603, 2024-09-16T14:07:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 49 2024-09-16T14:07:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0299999999999998e-07 2024-09-16T14:07:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226560+0000", 2024-09-16T14:07:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5879999999999999e-06 2024-09-16T14:07:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226568+0000", 2024-09-16T14:07:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3799999999999994e-06 2024-09-16T14:07:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226575+0000", 2024-09-16T14:07:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:07:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236676+0000", 2024-09-16T14:07:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010100448999999999 2024-09-16T14:07:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236851+0000", 2024-09-16T14:07:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017497899999999999 2024-09-16T14:07:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237123+0000", 2024-09-16T14:07:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027242299999999999 2024-09-16T14:07:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240139+0000", 2024-09-16T14:07:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030155109999999998 2024-09-16T14:07:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240368+0000", 2024-09-16T14:07:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000229461 2024-09-16T14:07:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240417+0000", 2024-09-16T14:07:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9045999999999998e-05 2024-09-16T14:07:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240492+0000", 2024-09-16T14:07:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5177000000000002e-05 2024-09-16T14:07:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.891480368000003, 2024-09-16T14:07:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015154025999999999, 2024-09-16T14:07:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 50 2024-09-16T14:07:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226710+0000", 2024-09-16T14:07:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012769999999999999 2024-09-16T14:07:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226721+0000", 2024-09-16T14:07:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-16T14:07:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226726+0000", 2024-09-16T14:07:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6459999999999998e-06 2024-09-16T14:07:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238128+0000", 2024-09-16T14:07:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011401390000000001 2024-09-16T14:07:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238249+0000", 2024-09-16T14:07:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121284 2024-09-16T14:07:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238407+0000", 2024-09-16T14:07:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015738899999999999 2024-09-16T14:07:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240509+0000", 2024-09-16T14:07:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002102923 2024-09-16T14:07:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241459+0000", 2024-09-16T14:07:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094996999999999996 2024-09-16T14:07:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241621+0000", 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000161502 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241736+0000", 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011458799999999999 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.891327443000002, 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015305664, 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 51 2024-09-16T14:07:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226735+0000", 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5700000000000001e-07 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226740+0000", 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6460000000000001e-06 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226748+0000", 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226753+0000", 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6229999999999996e-06 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239228+0000", 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012475119999999999 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239405+0000", 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177641 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239559+0000", 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015368599999999999 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241393+0000", 2024-09-16T14:07:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018340260000000001 2024-09-16T14:07:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241863+0000", 2024-09-16T14:07:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047025300000000001 2024-09-16T14:07:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241934+0000", 2024-09-16T14:07:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0229999999999997e-05 2024-09-16T14:07:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242040+0000", 2024-09-16T14:07:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106568 2024-09-16T14:07:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.891301048999999, 2024-09-16T14:07:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015737872, 2024-09-16T14:07:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 52 2024-09-16T14:07:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226762+0000", 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.23e-07 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226766+0000", 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.668e-06 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226774+0000", 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8939999999999993e-06 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226779+0000", 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1170000000000003e-06 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240584+0000", 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013804344 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240709+0000", 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012524300000000001 2024-09-16T14:07:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241014+0000", 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030534399999999998 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242062+0000", 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010479899999999999 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242369+0000", 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030710399999999999 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242442+0000", 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2383000000000001e-05 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242499+0000", 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.347040511000003, 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013869546999999999, 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 71 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771022+0000", 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0699999999999999e-06 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771031+0000", 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3899999999999993e-06 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771048+0000", 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6949e-05 2024-09-16T14:07:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771062+0000", 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4647000000000001e-05 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780217+0000", 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091543300000000005 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780323+0000", 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106223 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780485+0000", 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016238299999999999 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784807+0000", 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043215500000000004 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784821+0000", 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4268e-05 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784861+0000", 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9462000000000003e-05 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784891+0000", 2024-09-16T14:07:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0275000000000001e-05 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.346986065000003, 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015694993000000001, 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 72 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771077+0000", 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0299999999999997e-07 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771209+0000", 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132347 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771224+0000", 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.454e-05 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771230+0000", 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782272+0000", 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011041455 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782402+0000", 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013039099999999999 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782645+0000", 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024303500000000001 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786509+0000", 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003863698 2024-09-16T14:07:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786628+0000", 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011894499999999999 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786711+0000", 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2961999999999996e-05 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786771+0000", 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0098e-05 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.346823336, 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016692631999999999, 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 73 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771240+0000", 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.569999999999999e-07 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771244+0000", 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771253+0000", 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8629999999999997e-06 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771262+0000", 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783924+0000", 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012662479000000001 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784015+0000", 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0624999999999994e-05 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784129+0000", 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113563 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787598+0000", 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034694890000000001 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787707+0000", 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109341 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787840+0000", 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132421 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787931+0000", 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1291000000000004e-05 2024-09-16T14:07:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.346793239999997, 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017320314999999999, 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74 2024-09-16T14:07:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771278+0000", 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3790000000000006e-06 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771286+0000", 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5199999999999997e-06 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771292+0000", 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2839999999999997e-06 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784933+0000", 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013641304 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785019+0000", 2024-09-16T14:07:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6201999999999999e-05 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785164+0000", 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014457599999999999 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787609+0000", 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024456769999999998 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788403+0000", 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079383000000000003 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788490+0000", 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7231000000000006e-05 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788589+0000", 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8608000000000002e-05 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.346763529999997, 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017489688, 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771299+0000", 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3e-07 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771421+0000", 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121544 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771433+0000", 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853e-05 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771443+0000", 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0229999999999998e-06 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785601+0000", 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014158145 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785677+0000", 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6711999999999993e-05 2024-09-16T14:07:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785774+0000", 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6583999999999995e-05 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787633+0000", 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001859461 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788688+0000", 2024-09-16T14:07:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010546349999999999 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788747+0000", 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.923e-05 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788788+0000", 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0771e-05 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.346606950000002, 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017898397999999999, 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 76 2024-09-16T14:07:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771456+0000", 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7900000000000007e-07 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771466+0000", 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8800000000000003e-06 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771475+0000", 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1670000000000005e-06 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771480+0000", 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.49e-06 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786862+0000", 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015381807000000001 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786985+0000", 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012302099999999999 2024-09-16T14:07:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787142+0000", 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015696500000000001 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788157+0000", 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001014923 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789175+0000", 2024-09-16T14:07:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010175200000000001 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789259+0000", 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4578000000000002e-05 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789353+0000", 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4067999999999995e-05 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5397 3.1a 3:5f43d5fe:::rbd_data.1136494ba971.0000000000000020:7 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.39412351, 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013283031000000001, 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5397 2024-09-16T14:07:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723939+0000", 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.03e-07 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723941+0000", 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7910000000000001e-06 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723947+0000", 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.798e-06 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723951+0000", 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.546e-06 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737068+0000", 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013117199 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737147+0000", 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8473999999999994e-05 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737221+0000", 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4820000000000005e-05 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5470 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 73728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.3940003020000002, 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013300935, 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5470 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724064+0000", 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724069+0000", 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2050000000000001e-06 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724074+0000", 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7049999999999998e-06 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737243+0000", 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013169273 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737309+0000", 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6087000000000006e-05 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737363+0000", 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3341999999999999e-05 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5471 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.3939801109999994, 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013448857, 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5471 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3300000000000002e-07 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724085+0000", 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3460000000000001e-06 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724090+0000", 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7189999999999998e-06 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724094+0000", 2024-09-16T14:07:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-06 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737386+0000", 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013292438 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737461+0000", 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4554999999999993e-05 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737531+0000", 2024-09-16T14:07:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9726000000000003e-05 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5472 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.3939625160000002, 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013595701, 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5472 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724100+0000", 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8500000000000002e-07 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724108+0000", 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9759999999999995e-06 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724113+0000", 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6180000000000003e-06 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724117+0000", 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0550000000000001e-06 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737557+0000", 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013439184 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737633+0000", 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6682999999999997e-05 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737695+0000", 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1799999999999998e-05 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5757 3.1a 3:5c45bf46:::rbd_object_map.11369af4c40c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:32.236 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.3848845829999998, 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01336064, 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5757 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733178+0000", 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8100000000000003e-07 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733181+0000", 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4570000000000001e-06 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733188+0000", 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3589999999999996e-06 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733192+0000", 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0629999999999999e-06 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739596+0000", 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064040140000000004 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739669+0000", 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3559999999999994e-05 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.740040+0000", 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00037030500000000003 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.745915+0000", 2024-09-16T14:07:32.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058753859999999998 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746458+0000", 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054278499999999995 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746503+0000", 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4811999999999997e-05 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746538+0000", 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5417999999999999e-05 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:7540 3.10 3:0c0a7e4b:::rbd_data.11369af4c40c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.3113411960000008, 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016499916, 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7540 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806722+0000", 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.78e-07 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807847+0000", 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001125381 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807858+0000", 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1219999999999999e-05 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807863+0000", 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1569999999999999e-06 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808087+0000", 2024-09-16T14:07:32.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022387000000000001 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808219+0000", 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013168199999999999 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.809044+0000", 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082524200000000001 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.814222+0000", 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005177414 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823075+0000", 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088537470000000004 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823150+0000", 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4571000000000001e-05 2024-09-16T14:07:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823221+0000", 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0654000000000005e-05 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9727 3.1a 3:5b31ba1a:::rbd_data.1136a719415e.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.9067513720000004, 2024-09-16T14:07:32.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.033339056999999998, 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9727 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211313+0000", 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1969999999999999e-06 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216419+0000", 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051063580000000001 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216461+0000", 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2228000000000003e-05 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216473+0000", 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1635e-05 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216560+0000", 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6745999999999999e-05 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216730+0000", 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017044299999999999 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.217504+0000", 2024-09-16T14:07:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077432 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.232742+0000", 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015237611 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244490+0000", 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011747506 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244586+0000", 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6106000000000004e-05 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244650+0000", 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3906999999999996e-05 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9793 3.1a 3:5f64584f:::rbd_data.1136a719415e.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7696629509999999, 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024339903, 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9793 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:32.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348404+0000", 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.288e-06 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:32.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352536+0000", 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041315750000000002 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352576+0000", 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0309999999999999e-05 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352590+0000", 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3524e-05 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352690+0000", 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010055700000000001 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352873+0000", 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000182714 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.353759+0000", 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088597700000000003 2024-09-16T14:07:32.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.362898+0000", 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091386360000000003 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372631+0000", 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097337159999999999 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372700+0000", 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9016999999999993e-05 2024-09-16T14:07:32.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372739+0000", 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8588999999999999e-05 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:32.260 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:32.262 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:32.262 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:32.263 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:32.263 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:32.263 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:32.263 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:32.263 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:32.254+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:32.276 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:32.303 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:32.326 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:32.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:32.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:32.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:32.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:32.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:32.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:32.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:32.329 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:32.357 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:32.357 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:32.362 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:32.354+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:32.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:32.463 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:32.458+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:32.508 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:32.508 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:32.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:32.564 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:32.558+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:32.615 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:32.615 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:32.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:32.665 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:32.658+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:32.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:07:32.737 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:32.737 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:32.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:32.794 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:32.790+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:32.867 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:32.862+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:32.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:32.967 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:32.962+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.002 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.013 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:33.055 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-16T14:07:33.069 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:33.062+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:33.123 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.123 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:33.170 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:33.162+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:33.230 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.230 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.239 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:33.271 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:33.266+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:33.336 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.336 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.345 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:33.372 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:33.366+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:33.440 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.440 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:33.472 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:33.466+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:33.484 INFO:teuthology.orchestra.run.smithi079.stdout:1280 write 0xdae9d69 thru 0xdaf3e8f (0xa127 bytes) 2024-09-16T14:07:33.503 INFO:teuthology.orchestra.run.smithi079.stdout:1282 write 0xab9f209 thru 0xaba508c (0x5e84 bytes) 2024-09-16T14:07:33.511 INFO:teuthology.orchestra.run.smithi079.stdout:1283 punch from 0x5e9ec1 to 0x5f1b42, (0x7c81 bytes) 2024-09-16T14:07:33.512 INFO:teuthology.orchestra.run.smithi079.stdout:1284 write 0xad6cb83 thru 0xad727de (0x5c5c bytes) 2024-09-16T14:07:33.519 INFO:teuthology.orchestra.run.smithi079.stdout:1285 read 0x1c44082 thru 0x1c4d637 (0x95b6 bytes) 2024-09-16T14:07:33.520 INFO:teuthology.orchestra.run.smithi079.stdout:1287 writesame 0x9cae327 thru 0x9cba279 data_size 0x1(0xbf53 bytes) 2024-09-16T14:07:33.554 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.555 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:33.565 INFO:teuthology.orchestra.run.smithi079.stdout:1289 trunc from 0xde547bf to 0x9fe6e92 2024-09-16T14:07:33.573 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:33.566+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:33.588 INFO:teuthology.orchestra.run.smithi079.stdout:1291 read 0x24afea2 thru 0x24bea1f (0xeb7e bytes) 2024-09-16T14:07:33.589 INFO:teuthology.orchestra.run.smithi079.stdout:1292 write 0x576bac7 thru 0x576e6b0 (0x2bea bytes) 2024-09-16T14:07:33.598 INFO:teuthology.orchestra.run.smithi079.stdout:1294 read 0xd5d100 thru 0xd675a5 (0xa4a6 bytes) 2024-09-16T14:07:33.599 INFO:teuthology.orchestra.run.smithi079.stdout:1295 writesame 0x200536d thru 0x200b4b7 data_size 0x1(0x614b bytes) 2024-09-16T14:07:33.643 INFO:teuthology.orchestra.run.smithi079.stdout:1296 write 0xe2afc5d thru 0xe2bc372 (0xc716 bytes) 2024-09-16T14:07:33.650 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:33.650 INFO:teuthology.orchestra.run.smithi079.stdout:1298 read 0x15d4a72 thru 0x15e303f (0xe5ce bytes) 2024-09-16T14:07:33.665 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.665 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.670 INFO:teuthology.orchestra.run.smithi079.stdout:1299 trunc from 0xe2bc373 to 0x63889ce 2024-09-16T14:07:33.675 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:33.675 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:33.670+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:33.713 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:07:33.740 INFO:teuthology.orchestra.run.smithi079.stdout:1301 write 0xba299fd thru 0xba3612e (0xc732 bytes) 2024-09-16T14:07:33.752 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:33.752 INFO:teuthology.orchestra.run.smithi079.stdout:1303 punch from 0xa769a5c to 0xa76abdc, (0x1180 bytes) 2024-09-16T14:07:33.752 INFO:teuthology.orchestra.run.smithi079.stdout:1305 punch from 0x7bb9807 to 0x7bbc1ec, (0x29e5 bytes) 2024-09-16T14:07:33.752 INFO:teuthology.orchestra.run.smithi079.stdout:1307 writesame 0xfe0073 thru 0xfe8318 data_size 0x7(0x82a6 bytes) 2024-09-16T14:07:33.769 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.769 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.772 INFO:teuthology.orchestra.run.smithi079.stdout:1308 write 0xcf68b7f thru 0xcf6bb71 (0x2ff3 bytes) 2024-09-16T14:07:33.774 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:33.766+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:33.778 INFO:teuthology.orchestra.run.smithi079.stdout:1309 clone 10 order 18 su 8192 sc 10 2024-09-16T14:07:33.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:33.874 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.875 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.875 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:33.870+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:33.883 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:33.976 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:33.970+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:33.983 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:33.984 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:33.993 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:34.077 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:34.074+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:34.088 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:34.088 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:34.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:34.178 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:34.170+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:34.195 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:34.196 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:34.205 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:34.279 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:34.274+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:34.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:34.345 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:34.346 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:34.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:34.381 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:34.374+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:34.481 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:34.474+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:34.496 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:34.499 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:34.499 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:34.499 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:34.499 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:34.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:34.514 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:34.534 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:34.553 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:34.573 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:34.582 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:34.578+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:34.602 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:34.605 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:34.606 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:34.606 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:34.606 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:34.606 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:34.615 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:34.624 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:34.648 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:34.650 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:34.650 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:34.650 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:34.650 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:34.650 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:34.651 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:34.651 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:34.651 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:34.683 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:34.674+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:34.729 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:34.729 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:34.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:34.729 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:34.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.729 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4850 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.729 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.711267+0000", 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.014800887, 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014202039, 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4850 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711267+0000", 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711267+0000", 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711268+0000", 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8599999999999999e-07 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711269+0000", 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8449999999999999e-06 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711277+0000", 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2570000000000001e-06 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711281+0000", 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:07:34.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.722855+0000", 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011573685 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.722929+0000", 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399000000000004e-05 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725469+0000", 2024-09-16T14:07:34.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002540297 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4851 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.711286+0000", 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.014782568999999, 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014377928, 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4851 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711286+0000", 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711286+0000", 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711286+0000", 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711293+0000", 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8299999999999998e-06 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711302+0000", 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2860000000000005e-06 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.711307+0000", 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7389999999999999e-06 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725508+0000", 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014201266000000001 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725588+0000", 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0080999999999994e-05 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.725664+0000", 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5343e-05 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5053 3.1b 3:dd61974c:::rbd_data.1136494ba971.0000000000000074:7 [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715261+0000", 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010807209999999, 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01377916, 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5053 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715261+0000", 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715261+0000", 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715261+0000", 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-16T14:07:34.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715263+0000", 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715269+0000", 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9059999999999999e-06 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715273+0000", 2024-09-16T14:07:34.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2379999999999999e-06 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.728884+0000", 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013610344 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.728970+0000", 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6846000000000002e-05 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729040+0000", 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9599999999999998e-05 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5054 3.1b 3:dd61974c:::rbd_data.1136494ba971.0000000000000074:7 [read 196608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715278+0000", 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010790589999999, 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013927065000000001, 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5054 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715278+0000", 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715278+0000", 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715278+0000", 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4700000000000001e-07 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715280+0000", 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.024e-06 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715286+0000", 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2910000000000004e-06 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715290+0000", 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729066+0000", 2024-09-16T14:07:34.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013776149999999999 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729139+0000", 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2812999999999996e-05 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729205+0000", 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5314999999999994e-05 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.748 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5062 3.1b 3:dfaae061:::rbd_data.1136494ba971.000000000000007a:7 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.748 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715296+0000", 2024-09-16T14:07:34.748 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010772372, 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014091349, 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5062 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715296+0000", 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715296+0000", 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715296+0000", 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715298+0000", 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1919999999999999e-06 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715305+0000", 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6880000000000002e-06 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715309+0000", 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4470000000000002e-06 2024-09-16T14:07:34.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729229+0000", 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013920069 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729313+0000", 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4788000000000002e-05 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729387+0000", 2024-09-16T14:07:34.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3736e-05 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5063 3.1b 3:dfaae061:::rbd_data.1136494ba971.000000000000007a:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715313+0000", 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010754972000001, 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014351422000000001, 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5063 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715313+0000", 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715313+0000", 2024-09-16T14:07:34.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715314+0000", 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9800000000000003e-07 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715316+0000", 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0540000000000002e-06 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715322+0000", 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5320000000000003e-06 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715326+0000", 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7720000000000001e-06 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729523+0000", 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014197052 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729607+0000", 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3993999999999996e-05 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729665+0000", 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.732e-05 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.757 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5085 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 16384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010738104, 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015613336, 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5085 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0200000000000003e-07 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715333+0000", 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715339+0000", 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3629999999999999e-06 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715343+0000", 2024-09-16T14:07:34.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.924e-06 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729409+0000", 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014065846999999999 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.730871+0000", 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014620119999999999 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.730943+0000", 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2521000000000006e-05 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5086 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 98304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010721325, 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015853791999999998, 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5086 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3799999999999997e-07 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715349+0000", 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.136e-06 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715356+0000", 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1290000000000001e-06 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715359+0000", 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.495e-06 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731021+0000", 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015662367999999999 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731115+0000", 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3377999999999999e-05 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731201+0000", 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5748000000000006e-05 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5087 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010704928999999, 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016037577000000001, 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5087 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715364+0000", 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5100000000000001e-07 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715366+0000", 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715371+0000", 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3469999999999998e-06 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715374+0000", 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4450000000000001e-06 2024-09-16T14:07:34.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731249+0000", 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015874528999999998 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731332+0000", 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731401+0000", 2024-09-16T14:07:34.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9209999999999996e-05 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5097 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010639675, 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016148116000000001, 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5097 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715429+0000", 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0500000000000004e-07 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715432+0000", 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9569999999999998e-06 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715438+0000", 2024-09-16T14:07:34.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2519999999999996e-06 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715442+0000", 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8130000000000002e-06 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731427+0000", 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015984851000000001 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731498+0000", 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1260999999999994e-05 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731577+0000", 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8477000000000001e-05 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5098 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.776 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.010622230999999, 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016302106, 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5098 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:34.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715448+0000", 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.852e-06 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715455+0000", 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3720000000000001e-06 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715459+0000", 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0990000000000001e-06 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731602+0000", 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016142897 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731668+0000", 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6161999999999994e-05 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731748+0000", 2024-09-16T14:07:34.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0314000000000004e-05 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5311 3.1b 3:dfaae061:::rbd_data.1136494ba971.000000000000007a:7 [read 196608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723510+0000", 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002558494000001, 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014688708999999999, 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5311 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723510+0000", 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723510+0000", 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723510+0000", 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7899999999999999e-07 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723512+0000", 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7430000000000001e-06 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723518+0000", 2024-09-16T14:07:34.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7799999999999997e-06 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723522+0000", 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5179999999999998e-06 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738050+0000", 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014528078999999999 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738128+0000", 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8140999999999996e-05 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738198+0000", 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0068999999999996e-05 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.785 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5349 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723619+0000", 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002448680000001, 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013772769000000001, 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5349 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723619+0000", 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723619+0000", 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723620+0000", 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723621+0000", 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6470000000000001e-06 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723627+0000", 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.75e-06 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723631+0000", 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:07:34.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737228+0000", 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013596949000000001 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737305+0000", 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6516999999999999e-05 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737392+0000", 2024-09-16T14:07:34.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7682999999999994e-05 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5350 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723635+0000", 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002433105, 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013958325000000001, 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5350 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723635+0000", 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723635+0000", 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723636+0000", 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5899999999999996e-07 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723640+0000", 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3789999999999999e-06 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723646+0000", 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8919999999999997e-06 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723651+0000", 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2240000000000003e-06 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737414+0000", 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013763266 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737510+0000", 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5483999999999996e-05 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737593+0000", 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3421e-05 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5421 3.3 3:c5757882:::rbd_data.1136494ba971.000000000000002c:7 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.794 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723848+0000", 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002220496, 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013924826, 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5421 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723848+0000", 2024-09-16T14:07:34.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723848+0000", 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723848+0000", 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723850+0000", 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6959999999999999e-06 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723856+0000", 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1059999999999996e-06 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723860+0000", 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:07:34.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737616+0000", 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013756286 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737691+0000", 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5277000000000004e-05 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737772+0000", 2024-09-16T14:07:34.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1213999999999999e-05 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5422 3.3 3:c5757882:::rbd_data.1136494ba971.000000000000002c:7 [read 81920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723864+0000", 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002204581999999, 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014087064, 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5422 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723864+0000", 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723864+0000", 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723864+0000", 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723866+0000", 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.846e-06 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723872+0000", 2024-09-16T14:07:34.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1299999999999998e-06 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723876+0000", 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7100000000000001e-06 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737794+0000", 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013918363 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737885+0000", 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0586000000000003e-05 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737951+0000", 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6021999999999999e-05 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5423 3.3 3:c5757882:::rbd_data.1136494ba971.000000000000002c:7 [read 172032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723882+0000", 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002186093000001, 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014510959, 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5423 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723882+0000", 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723882+0000", 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723882+0000", 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723884+0000", 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.652e-06 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723891+0000", 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6830000000000002e-06 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723895+0000", 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:07:34.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737973+0000", 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014078146999999999 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738279+0000", 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030630399999999997 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738393+0000", 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113818 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5424 3.3 3:c5757882:::rbd_data.1136494ba971.000000000000002c:7 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723899+0000", 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002169460999999, 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015007779000000001, 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5424 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723899+0000", 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723899+0000", 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723899+0000", 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723901+0000", 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8700000000000001e-06 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723907+0000", 2024-09-16T14:07:34.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0179999999999998e-06 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723911+0000", 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8909999999999997e-06 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738581+0000", 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014670337 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738656+0000", 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4598000000000006e-05 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738906+0000", 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025068099999999999 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5461 3.1b 3:d9580610:::rbd_data.1136494ba971.000000000000003a:7 [read 73728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.812 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723915+0000", 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002153453, 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014811758, 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5461 2024-09-16T14:07:34.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723915+0000", 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723915+0000", 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723915+0000", 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723922+0000", 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9879999999999998e-06 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723929+0000", 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4570000000000002e-06 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723932+0000", 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5939999999999998e-06 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738418+0000", 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014485736000000001 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738495+0000", 2024-09-16T14:07:34.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6792999999999995e-05 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738726+0000", 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023179400000000001 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5462 3.1b 3:d9580610:::rbd_data.1136494ba971.000000000000003a:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723936+0000", 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.002131705, 2024-09-16T14:07:34.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014951133, 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5462 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723936+0000", 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723936+0000", 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723937+0000", 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.03e-07 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723939+0000", 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9300000000000002e-06 2024-09-16T14:07:34.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723945+0000", 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0560000000000001e-06 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723949+0000", 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7409999999999998e-06 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:34.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738748+0000", 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014799909999999999 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738822+0000", 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3542999999999997e-05 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.738888+0000", 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5549999999999994e-05 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:34.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:34.822 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:34.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:34.824 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone9 from 0xcf6bb72 (overlap 0x63889ce) to 0x59948d3 2024-09-16T14:07:34.824 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:34.818+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:34.922 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:34.922 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:34.922 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:34.923 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:34.918+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:34.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:34.981 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-16T14:07:35.024 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:35.018+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:35.034 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:35.034 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:35.034 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:35.034 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:35.034 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:35.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:35.124 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:35.118+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:35.153 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:35.153 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:35.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:35.153 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:35.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.301123941999997, 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15070547300000001, 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847226+0000", 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.204e-06 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847235+0000", 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847250+0000", 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5662e-05 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847261+0000", 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0236e-05 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847327+0000", 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.622e-05 2024-09-16T14:07:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847345+0000", 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8434e-05 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.993706+0000", 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14636041899999999 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994117+0000", 2024-09-16T14:07:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041132499999999998 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994428+0000", 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031060899999999999 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.995845+0000", 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014173359999999999 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997768+0000", 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001922991 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997867+0000", 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9057999999999999e-05 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997929+0000", 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2433000000000003e-05 2024-09-16T14:07:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.921872309000001, 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013812964, 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 48 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226476+0000", 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226477+0000", 2024-09-16T14:07:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.879999999999999e-07 2024-09-16T14:07:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226482+0000", 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0919999999999997e-06 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226537+0000", 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4916000000000001e-05 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226546+0000", 2024-09-16T14:07:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9709999999999999e-06 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236148+0000", 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0096011849999999999 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236208+0000", 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9922e-05 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236308+0000", 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010068000000000001 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239183+0000", 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002874779 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240208+0000", 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001025312 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240255+0000", 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6363999999999999e-05 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240288+0000", 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3754999999999998e-05 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.921793207999997, 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013937603, 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 49 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.168 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226555+0000", 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0299999999999998e-07 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226560+0000", 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5879999999999999e-06 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226568+0000", 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3799999999999994e-06 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226575+0000", 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236676+0000", 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010100448999999999 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.236851+0000", 2024-09-16T14:07:35.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017497899999999999 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.237123+0000", 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027242299999999999 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240139+0000", 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030155109999999998 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240368+0000", 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000229461 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240417+0000", 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9045999999999998e-05 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240492+0000", 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5177000000000002e-05 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.921766320000003, 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015154025999999999, 2024-09-16T14:07:35.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 50 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226710+0000", 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012769999999999999 2024-09-16T14:07:35.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226721+0000", 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226726+0000", 2024-09-16T14:07:35.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6459999999999998e-06 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238128+0000", 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011401390000000001 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238249+0000", 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121284 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238407+0000", 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015738899999999999 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240509+0000", 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002102923 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241459+0000", 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094996999999999996 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241621+0000", 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000161502 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241736+0000", 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011458799999999999 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.921613395000001, 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015305664, 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 51 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.181 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226735+0000", 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5700000000000001e-07 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226740+0000", 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6460000000000001e-06 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226748+0000", 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226753+0000", 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6229999999999996e-06 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239228+0000", 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012475119999999999 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239405+0000", 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177641 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239559+0000", 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015368599999999999 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241393+0000", 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018340260000000001 2024-09-16T14:07:35.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241863+0000", 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047025300000000001 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241934+0000", 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0229999999999997e-05 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242040+0000", 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106568 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.921587000999999, 2024-09-16T14:07:35.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015737872, 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 52 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:35.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226762+0000", 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.23e-07 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226766+0000", 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.668e-06 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226774+0000", 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8939999999999993e-06 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226779+0000", 2024-09-16T14:07:35.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1170000000000003e-06 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240584+0000", 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013804344 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240709+0000", 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012524300000000001 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241014+0000", 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030534399999999998 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242062+0000", 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010479899999999999 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242369+0000", 2024-09-16T14:07:35.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030710399999999999 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242442+0000", 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2383000000000001e-05 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242499+0000", 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.377326463000003, 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013869546999999999, 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.194 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 71 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771021+0000", 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771022+0000", 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0699999999999999e-06 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771031+0000", 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3899999999999993e-06 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771048+0000", 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6949e-05 2024-09-16T14:07:35.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771062+0000", 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4647000000000001e-05 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780217+0000", 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091543300000000005 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780323+0000", 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106223 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.780485+0000", 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016238299999999999 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784807+0000", 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043215500000000004 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784821+0000", 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4268e-05 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784861+0000", 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9462000000000003e-05 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784891+0000", 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0275000000000001e-05 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.377272017000003, 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015694993000000001, 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.200 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 72 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771077+0000", 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0299999999999997e-07 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771209+0000", 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132347 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771224+0000", 2024-09-16T14:07:35.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.454e-05 2024-09-16T14:07:35.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771230+0000", 2024-09-16T14:07:35.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782272+0000", 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011041455 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782402+0000", 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013039099999999999 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782645+0000", 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024303500000000001 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786509+0000", 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003863698 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786628+0000", 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011894499999999999 2024-09-16T14:07:35.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786711+0000", 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2961999999999996e-05 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786771+0000", 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0098e-05 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.377109288, 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016692631999999999, 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 73 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.207 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771240+0000", 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.569999999999999e-07 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771244+0000", 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771253+0000", 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8629999999999997e-06 2024-09-16T14:07:35.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771262+0000", 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783924+0000", 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012662479000000001 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784015+0000", 2024-09-16T14:07:35.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0624999999999994e-05 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784129+0000", 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113563 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787598+0000", 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034694890000000001 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787707+0000", 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109341 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787840+0000", 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132421 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787931+0000", 2024-09-16T14:07:35.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1291000000000004e-05 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.377079191999997, 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017320314999999999, 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:35.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771278+0000", 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3790000000000006e-06 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771286+0000", 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5199999999999997e-06 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771292+0000", 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2839999999999997e-06 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784933+0000", 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013641304 2024-09-16T14:07:35.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785019+0000", 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6201999999999999e-05 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785164+0000", 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014457599999999999 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787609+0000", 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024456769999999998 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788403+0000", 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079383000000000003 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788490+0000", 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7231000000000006e-05 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788589+0000", 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8608000000000002e-05 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.377049481999997, 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017489688, 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771299+0000", 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3e-07 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771421+0000", 2024-09-16T14:07:35.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121544 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771433+0000", 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853e-05 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771443+0000", 2024-09-16T14:07:35.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0229999999999998e-06 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785601+0000", 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014158145 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785677+0000", 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6711999999999993e-05 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785774+0000", 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6583999999999995e-05 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787633+0000", 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001859461 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788688+0000", 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010546349999999999 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788747+0000", 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.923e-05 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788788+0000", 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0771e-05 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.225 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.376892902000002, 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017898397999999999, 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:35.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 76 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771456+0000", 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7900000000000007e-07 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771466+0000", 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8800000000000003e-06 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771475+0000", 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1670000000000005e-06 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771480+0000", 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.49e-06 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786862+0000", 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015381807000000001 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786985+0000", 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012302099999999999 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787142+0000", 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015696500000000001 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788157+0000", 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001014923 2024-09-16T14:07:35.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789175+0000", 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010175200000000001 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789259+0000", 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4578000000000002e-05 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789353+0000", 2024-09-16T14:07:35.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4067999999999995e-05 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5397 3.1a 3:5f43d5fe:::rbd_data.1136494ba971.0000000000000020:7 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.424409462, 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013283031000000001, 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:35.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5397 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723938+0000", 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723939+0000", 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.03e-07 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723941+0000", 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7910000000000001e-06 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723947+0000", 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.798e-06 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.723951+0000", 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.546e-06 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737068+0000", 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013117199 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737147+0000", 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8473999999999994e-05 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737221+0000", 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4820000000000005e-05 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5470 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 73728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.236 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.424286254, 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013300935, 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5470 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724062+0000", 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724064+0000", 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724069+0000", 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2050000000000001e-06 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724074+0000", 2024-09-16T14:07:35.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7049999999999998e-06 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737243+0000", 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013169273 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737309+0000", 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6087000000000006e-05 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737363+0000", 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3341999999999999e-05 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5471 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.424266062999999, 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013448857, 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:35.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5471 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724082+0000", 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3300000000000002e-07 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724085+0000", 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3460000000000001e-06 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724090+0000", 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7189999999999998e-06 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724094+0000", 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4400000000000001e-06 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737386+0000", 2024-09-16T14:07:35.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013292438 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737461+0000", 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4554999999999993e-05 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737531+0000", 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9726000000000003e-05 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5472 3.1a 3:58df3adb:::rbd_data.1136494ba971.000000000000003f:7 [read 245760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.424248468, 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013595701, 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5472 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.246 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724099+0000", 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724100+0000", 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8500000000000002e-07 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724108+0000", 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9759999999999995e-06 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724113+0000", 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6180000000000003e-06 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724117+0000", 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0550000000000001e-06 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737557+0000", 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013439184 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737633+0000", 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6682999999999997e-05 2024-09-16T14:07:35.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.737695+0000", 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1799999999999998e-05 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5757 3.1a 3:5c45bf46:::rbd_object_map.11369af4c40c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.415170535, 2024-09-16T14:07:35.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01336064, 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5757 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733177+0000", 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733178+0000", 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8100000000000003e-07 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733181+0000", 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4570000000000001e-06 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733188+0000", 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3589999999999996e-06 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.733192+0000", 2024-09-16T14:07:35.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0629999999999999e-06 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739596+0000", 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0064040140000000004 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.739669+0000", 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3559999999999994e-05 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.740040+0000", 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00037030500000000003 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.745915+0000", 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058753859999999998 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746458+0000", 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054278499999999995 2024-09-16T14:07:35.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746503+0000", 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4811999999999997e-05 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.746538+0000", 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5417999999999999e-05 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:7540 3.10 3:0c0a7e4b:::rbd_data.11369af4c40c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.341627148000001, 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016499916, 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7540 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.257 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806722+0000", 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.78e-07 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807847+0000", 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001125381 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807858+0000", 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1219999999999999e-05 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807863+0000", 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1569999999999999e-06 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808087+0000", 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022387000000000001 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808219+0000", 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013168199999999999 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.809044+0000", 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082524200000000001 2024-09-16T14:07:35.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.814222+0000", 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005177414 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823075+0000", 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088537470000000004 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823150+0000", 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4571000000000001e-05 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823221+0000", 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0654000000000005e-05 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9727 3.1a 3:5b31ba1a:::rbd_data.1136a719415e.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9370373240000003, 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.033339056999999998, 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9727 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211313+0000", 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1969999999999999e-06 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216419+0000", 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051063580000000001 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216461+0000", 2024-09-16T14:07:35.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2228000000000003e-05 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216473+0000", 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1635e-05 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216560+0000", 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6745999999999999e-05 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216730+0000", 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017044299999999999 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.217504+0000", 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077432 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.232742+0000", 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015237611 2024-09-16T14:07:35.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244490+0000", 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011747506 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244586+0000", 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6106000000000004e-05 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244650+0000", 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3906999999999996e-05 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9793 3.1a 3:5f64584f:::rbd_data.1136a719415e.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.7999489029999998, 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024339903, 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9793 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348404+0000", 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.288e-06 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352536+0000", 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041315750000000002 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352576+0000", 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0309999999999999e-05 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352590+0000", 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3524e-05 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352690+0000", 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010055700000000001 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352873+0000", 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000182714 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.353759+0000", 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088597700000000003 2024-09-16T14:07:35.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.362898+0000", 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091386360000000003 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372631+0000", 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097337159999999999 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372700+0000", 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9016999999999993e-05 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372739+0000", 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8588999999999999e-05 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.276 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:35.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:35.278 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:35.274+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:35.376 INFO:teuthology.orchestra.run.smithi079.stdout:1310 writesame 0x58509be thru 0x58591ee data_size 0x13(0x8831 bytes) 2024-09-16T14:07:35.378 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:35.374+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:35.382 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:35.382 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:35.382 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:35.382 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:35.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:35.404 INFO:teuthology.orchestra.run.smithi079.stdout:1311 trunc from 0xcf6bb72 to 0xc54db69 2024-09-16T14:07:35.428 INFO:teuthology.orchestra.run.smithi079.stdout:1312 punch from 0x1361320 to 0x1362d7a, (0x1a5a bytes) 2024-09-16T14:07:35.428 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:35.428 INFO:teuthology.orchestra.run.smithi079.stdout:1315 write 0x81c23a7 thru 0x81c4079 (0x1cd3 bytes) 2024-09-16T14:07:35.434 INFO:teuthology.orchestra.run.smithi079.stdout:1316 write 0x302c83c thru 0x302f9e0 (0x31a5 bytes) 2024-09-16T14:07:35.441 INFO:teuthology.orchestra.run.smithi079.stdout:1317 read 0xb65600b thru 0xb66351f (0xd515 bytes) 2024-09-16T14:07:35.441 INFO:teuthology.orchestra.run.smithi079.stdout:1319 write 0x3276421 thru 0x327da11 (0x75f1 bytes) 2024-09-16T14:07:35.460 INFO:teuthology.orchestra.run.smithi079.stdout:1320 write 0x7f4be55 thru 0x7f55417 (0x95c3 bytes) 2024-09-16T14:07:35.471 INFO:teuthology.orchestra.run.smithi079.stdout:1321 punch from 0xbf4670f to 0xbf4e25c, (0x7b4d bytes) 2024-09-16T14:07:35.471 INFO:teuthology.orchestra.run.smithi079.stdout:1322 writesame 0x5745af5 thru 0x5754e57 data_size 0xa1(0xf363 bytes) 2024-09-16T14:07:35.480 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:35.474+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:35.489 INFO:teuthology.orchestra.run.smithi079.stdout:1323 read 0xb648747 thru 0xb64af25 (0x27df bytes) 2024-09-16T14:07:35.490 INFO:teuthology.orchestra.run.smithi079.stdout:1326 punch from 0x58fa502 to 0x58ff538, (0x5036 bytes) 2024-09-16T14:07:35.499 INFO:teuthology.orchestra.run.smithi079.stdout:1328 write 0xd6de298 thru 0xd6ea1d0 (0xbf39 bytes) 2024-09-16T14:07:35.512 INFO:teuthology.orchestra.run.smithi079.stdout:1329 read 0x19ea69f thru 0x19f4ab7 (0xa419 bytes) 2024-09-16T14:07:35.512 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:35.513 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:35.513 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:35.513 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:35.513 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:35.515 INFO:teuthology.orchestra.run.smithi079.stdout:1331 write 0xa6c0b2c thru 0xa6cd9db (0xceb0 bytes) 2024-09-16T14:07:35.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:35.553 INFO:teuthology.orchestra.run.smithi079.stdout:1332 write 0xee0ad9d thru 0xee12e90 (0x80f4 bytes) 2024-09-16T14:07:35.568 INFO:teuthology.orchestra.run.smithi079.stdout:1333 write 0x715a560 thru 0x71606de (0x617f bytes) 2024-09-16T14:07:35.580 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:35.574+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:35.584 INFO:teuthology.orchestra.run.smithi079.stdout:1335 trunc from 0xee12e91 to 0xc3a6274 2024-09-16T14:07:35.606 INFO:teuthology.orchestra.run.smithi079.stdout:1336 writesame 0x972a70e thru 0x973362b data_size 0x7(0x8f1e bytes) 2024-09-16T14:07:35.617 INFO:teuthology.orchestra.run.smithi079.stdout:1338 read 0x8eecd thru 0x9d761 (0xe895 bytes) 2024-09-16T14:07:35.621 INFO:teuthology.orchestra.run.smithi079.stdout:1341 read 0x43bc21e thru 0x43bcdbf (0xba2 bytes) 2024-09-16T14:07:35.656 INFO:teuthology.orchestra.run.smithi079.stdout:1342 write 0xe3e54e1 thru 0xe3f2adc (0xd5fc bytes) 2024-09-16T14:07:35.670 INFO:teuthology.orchestra.run.smithi079.stdout:1345 write 0x1ad2fa thru 0x1ba8fd (0xd604 bytes) 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.313737462999999, 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056564060000000001, 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357785+0000", 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0779999999999998e-06 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357794+0000", 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357811+0000", 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7300999999999999e-05 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357825+0000", 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3959e-05 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357934+0000", 2024-09-16T14:07:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010899 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357942+0000", 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.182e-06 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.360955+0000", 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003013258 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361149+0000", 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019401600000000001 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361448+0000", 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029863300000000003 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.362788+0000", 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013406480000000001 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363338+0000", 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054941500000000004 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363400+0000", 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2440999999999994e-05 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363440+0000", 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9734999999999998e-05 2024-09-16T14:07:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:694 3.e 3:76483abd:::rbd_data.1136e0d3367d.000000000000016c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.897105972999999, 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055514199999999996, 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 694 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774417+0000", 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0509999999999999e-06 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774424+0000", 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4229999999999999e-06 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774443+0000", 2024-09-16T14:07:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8664999999999998e-05 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774457+0000", 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3974e-05 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774531+0000", 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4208999999999994e-05 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774612+0000", 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0438000000000005e-05 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.775728+0000", 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001116414 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.777045+0000", 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013172139999999999 2024-09-16T14:07:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779908+0000", 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002863071 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779939+0000", 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0840000000000003e-05 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779966+0000", 2024-09-16T14:07:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7121e-05 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:1028 3.5 3:a45e2964:::rbd_data.11365ca8c0d.000000000000003b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.515376890999999, 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051968600000000002, 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1028 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156144+0000", 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156146+0000", 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5129999999999999e-06 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156159+0000", 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.276e-05 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156189+0000", 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0209000000000002e-05 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156210+0000", 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0647000000000001e-05 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156339+0000", 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012944000000000001 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.156648+0000", 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030876299999999998 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.161341+0000", 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046925279999999996 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1106 3.11 e30/23 3:8da88c9c:::rbd_data.11365ca8c0d.0000000000000014:head v 30'5, mlcod=28'1)", 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.412599632999999, 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054785859999999997, 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258922+0000", 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9500000000000001e-07 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258929+0000", 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4560000000000003e-06 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258944+0000", 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4548e-05 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258950+0000", 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9959999999999999e-06 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.259023+0000", 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2805000000000005e-05 2024-09-16T14:07:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.259056+0000", 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3395000000000001e-05 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.264352+0000", 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052958019999999996 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.264400+0000", 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7788999999999999e-05 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:3850 3.16 3:6fa6863f:::rbd_data.1136494ba971.000000000000021b:head [set-alloc-hint object_size 262144 write_size 262144,write 58251~7285] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.290889582, 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052342250000000003, 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3850 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380633+0000", 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8920000000000001e-06 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380647+0000", 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3455000000000001e-05 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380670+0000", 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3215999999999999e-05 2024-09-16T14:07:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380678+0000", 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2779999999999993e-06 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380734+0000", 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6255999999999998e-05 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380799+0000", 2024-09-16T14:07:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4204999999999999e-05 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380900+0000", 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101672 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385645+0000", 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047443859999999997 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385755+0000", 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110315 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385815+0000", 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0035999999999999e-05 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385865+0000", 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0513999999999998e-05 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:4098 3.1e e35/23 3:783d22fa:::rbd_data.1136494ba971.0000000000000017:head v 35'28, mlcod=35'22)", 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.247247172, 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00715775, 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6499999999999998e-07 2024-09-16T14:07:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424283+0000", 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6840000000000002e-06 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424295+0000", 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2074e-05 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424302+0000", 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.489e-06 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424345+0000", 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2846999999999998e-05 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424364+0000", 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9034000000000001e-05 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431403+0000", 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070398259999999999 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431431+0000", 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8031e-05 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4102 3.16 3:6ee147bb:::rbd_data.1136494ba971.000000000000001b:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.246483326, 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063615549999999996, 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4102 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425038+0000", 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0400000000000005e-07 2024-09-16T14:07:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425042+0000", 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9090000000000002e-06 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425056+0000", 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3308e-05 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425060+0000", 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7820000000000001e-06 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425105+0000", 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4833000000000003e-05 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425174+0000", 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9125e-05 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425313+0000", 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013855900000000001 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426552+0000", 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00123953 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431313+0000", 2024-09-16T14:07:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004760401 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431361+0000", 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8535999999999997e-05 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431399+0000", 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7667999999999999e-05 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4105 3.5 3:a4636173:::rbd_data.1136494ba971.000000000000001e:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.244787913, 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078170070000000008, 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4105 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426734+0000", 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.088e-06 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426739+0000", 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8550000000000001e-06 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426749+0000", 2024-09-16T14:07:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0336999999999999e-05 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426755+0000", 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426798+0000", 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2579e-05 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426862+0000", 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3493999999999999e-05 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426974+0000", 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112271 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.429506+0000", 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025319439999999999 2024-09-16T14:07:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434430+0000", 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049244240000000002 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434501+0000", 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1023000000000001e-05 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434550+0000", 2024-09-16T14:07:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8678999999999999e-05 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4127 3.5 3:a4205384:::rbd_data.1136494ba971.0000000000000032:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.234802438999999, 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054291670000000004, 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4127 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436719+0000", 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9999999999999995e-07 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436724+0000", 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436739+0000", 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.482e-05 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436744+0000", 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.775e-06 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436791+0000", 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7420000000000003e-05 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436886+0000", 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4942e-05 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437094+0000", 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020822399999999999 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.439088+0000", 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019939340000000002 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.441989+0000", 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029012360000000002 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.442047+0000", 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:07:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.442148+0000", 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010031600000000001 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4161 3.2 3:467b8b19:::rbd_data.1136494ba971.000000000000005a:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.217290687, 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054131559999999997, 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4161 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454232+0000", 2024-09-16T14:07:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.756e-06 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454239+0000", 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9110000000000004e-06 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454259+0000", 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9749999999999999e-05 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454266+0000", 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8569999999999995e-06 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454334+0000", 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7104e-05 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454500+0000", 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016668100000000001 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454703+0000", 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020301299999999999 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.457949+0000", 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032455309999999999 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459554+0000", 2024-09-16T14:07:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016048 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459599+0000", 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5089999999999997e-05 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459643+0000", 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4663000000000003e-05 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5304 3.e 3:73080399:::rbd_data.1136494ba971.0000000000000075:7 [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.952561137, 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052879110000000002, 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5304 2024-09-16T14:07:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1800000000000001e-07 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718962+0000", 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.03e-06 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718969+0000", 2024-09-16T14:07:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2490000000000004e-06 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718983+0000", 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3145999999999999e-05 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724107+0000", 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051242450000000004 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724185+0000", 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8077999999999993e-05 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724248+0000", 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2744999999999996e-05 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9030 3.16 3:6f8d7c1d:::rbd_data.1136dd12cea0.0000000000000027:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.8530820390000002, 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082377660000000005, 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9030 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818442+0000", 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7669999999999999e-06 2024-09-16T14:07:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818452+0000", 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0763e-05 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818486+0000", 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3942000000000001e-05 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818499+0000", 2024-09-16T14:07:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2571e-05 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818602+0000", 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010291200000000001 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818781+0000", 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017939099999999999 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.819167+0000", 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038538 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.822000+0000", 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028333490000000002 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826582+0000", 2024-09-16T14:07:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045822989999999997 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826632+0000", 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9780000000000001e-05 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826677+0000", 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4611999999999999e-05 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9432 3.5 3:a779edfa:::rbd_data.11362e0d5ebf.000000000000008f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:26.854698+0000", 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.816822964, 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053152340000000003, 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9432 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854698+0000", 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854698+0000", 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854699+0000", 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.618e-06 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854708+0000", 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8559999999999996e-06 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854726+0000", 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7342000000000001e-05 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854735+0000", 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8559999999999996e-06 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854828+0000", 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3232000000000005e-05 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854949+0000", 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012077400000000001 2024-09-16T14:07:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.855237+0000", 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028819400000000003 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.856832+0000", 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001595015 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.859952+0000", 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031203849999999998 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.859991+0000", 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8430000000000003e-05 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.860013+0000", 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2532e-05 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9433 3.5 3:a3d6a849:::rbd_data.11362e0d5ebf.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:26.854753+0000", 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.8167680700000002, 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057192140000000002, 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9433 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854753+0000", 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854753+0000", 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854753+0000", 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9699999999999995e-07 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854758+0000", 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1160000000000001e-06 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854770+0000", 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.222e-05 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854776+0000", 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7769999999999999e-06 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.855664+0000", 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088838700000000003 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.855758+0000", 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4233000000000004e-05 2024-09-16T14:07:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.856044+0000", 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028594900000000002 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.857128+0000", 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010842530000000001 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.860379+0000", 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032508179999999999 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.860433+0000", 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3594999999999998e-05 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.860472+0000", 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9168999999999997e-05 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9465 3.5 3:a7744965:::rbd_object_map.11362e0d5ebf.000000000000000a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:26.863709+0000", 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.8078118280000002, 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066737389999999997, 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9465 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863709+0000", 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863709+0000", 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863710+0000", 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0200000000000002e-07 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863713+0000", 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3160000000000001e-06 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863727+0000", 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3753e-05 2024-09-16T14:07:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863733+0000", 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1759999999999998e-06 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.864597+0000", 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086442099999999998 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.864645+0000", 2024-09-16T14:07:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7250000000000003e-05 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.864899+0000", 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025424900000000001 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.865641+0000", 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074190900000000001 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.870288+0000", 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046474639999999996 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.870340+0000", 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2157000000000002e-05 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.870383+0000", 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2441999999999998e-05 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:9863 3.11 e39/23 3:8c69c14f:::rbd_data.1136a719415e.000000000000000f:head v 39'37, mlcod=37'34)", 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.2790504600000006, 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081439630000000006, 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392472+0000", 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7239999999999999e-06 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395095+0000", 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002622943 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395116+0000", 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0784e-05 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395127+0000", 2024-09-16T14:07:35.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0838e-05 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395189+0000", 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2211000000000005e-05 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395228+0000", 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8695000000000001e-05 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.400576+0000", 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053489289999999997 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.400614+0000", 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7839000000000001e-05 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13208 3.5 3:a0deb963:::rbd_data.11361fa8f1cb.000000000000016c:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:35.766 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.806582+0000", 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.86493845300000005, 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067377100000000001, 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13208 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806582+0000", 2024-09-16T14:07:35.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806582+0000", 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806585+0000", 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806597+0000", 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2554e-05 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806639+0000", 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1427999999999997e-05 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806656+0000", 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6956000000000002e-05 2024-09-16T14:07:35.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806759+0000", 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103812 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.807107+0000", 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034792800000000001 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813320+0000", 2024-09-16T14:07:35.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062127440000000001 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13209 3.5 3:a693b338:::rbd_data.11361fa8f1cb.000000000000016d:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.86483022399999998, 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086662360000000008, 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13209 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806692+0000", 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.345e-06 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806699+0000", 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8399999999999997e-06 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806718+0000", 2024-09-16T14:07:35.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8828e-05 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806729+0000", 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1069e-05 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.807862+0000", 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011332429999999999 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.808093+0000", 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023125700000000001 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815357+0000", 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072636539999999996 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13221 3.5 3:a0deb963:::rbd_data.11361fa8f1cb.000000000000016c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.813642+0000", 2024-09-16T14:07:35.775 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.85787836100000003, 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062358709999999996, 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13221 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813642+0000", 2024-09-16T14:07:35.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813642+0000", 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813644+0000", 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2500000000000001e-06 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813651+0000", 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1539999999999996e-06 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813669+0000", 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8547999999999999e-05 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813680+0000", 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0266e-05 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813802+0000", 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121883 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.814023+0000", 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000221249 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.814574+0000", 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00055117899999999995 2024-09-16T14:07:35.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815599+0000", 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010245969999999999 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.819794+0000", 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004194985 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.819834+0000", 2024-09-16T14:07:35.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0336999999999997e-05 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.819878+0000", 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4422999999999998e-05 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13226 3.5 3:a693b338:::rbd_data.11361fa8f1cb.000000000000016d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.815618+0000", 2024-09-16T14:07:35.781 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.85590257400000003, 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067527730000000001, 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13226 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815618+0000", 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815618+0000", 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815620+0000", 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.426e-06 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815629+0000", 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8380000000000008e-06 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815651+0000", 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2768e-05 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815661+0000", 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0013999999999999e-05 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815738+0000", 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7058000000000007e-05 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815836+0000", 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7843000000000005e-05 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815994+0000", 2024-09-16T14:07:35.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015771099999999999 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.816884+0000", 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088978199999999999 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.822285+0000", 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005401004 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:35.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.822335+0000", 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0714999999999998e-05 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.822371+0000", 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5614000000000002e-05 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:35.787 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:35.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:07:35.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:1346 writesame 0x4fda5c8 thru 0x4fde03c data_size 0x29(0x3a75 bytes) 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:1347 read 0x1353c33 thru 0x13596d2 (0x5aa0 bytes) 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:1348 read 0x5d2db24 thru 0x5d392a5 (0xb782 bytes) 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:1350 punch from 0x82f201e to 0x82f38f0, (0x18d2 bytes) 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:1355 read 0x2aad976 thru 0x2ab1e93 (0x451e bytes) 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:1356 read 0x2fa8788 thru 0x2fb095c (0x81d5 bytes) 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:1359 punch from 0x1fe2214 to 0x1fe6ca8, (0x4a94 bytes) 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:1361 write 0xd265bfb thru 0xd26b3eb (0x57f1 bytes) 2024-09-16T14:07:35.791 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:35.792 INFO:teuthology.orchestra.run.smithi079.stdout:1363 write 0x2b2bc29 thru 0x2b36945 (0xad1d bytes) 2024-09-16T14:07:35.792 INFO:teuthology.orchestra.run.smithi079.stdout:1364 writesame 0x21df7c6 thru 0x21df81c data_size 0x1(0x57 bytes) 2024-09-16T14:07:35.792 INFO:teuthology.orchestra.run.smithi079.stdout:1365 write 0x57b1d4c thru 0x57bd568 (0xb81d bytes) 2024-09-16T14:07:35.792 INFO:teuthology.orchestra.run.smithi079.stdout:1366 write 0x7cefde1 thru 0x7cf9968 (0x9b88 bytes) 2024-09-16T14:07:35.792 INFO:teuthology.orchestra.run.smithi079.stdout:1367 read 0x188d829 thru 0x189aa44 (0xd21c bytes) 2024-09-16T14:07:35.792 INFO:teuthology.orchestra.run.smithi079.stdout:1368 write 0xcf981a2 thru 0xcfa792c (0xf78b bytes) 2024-09-16T14:07:35.792 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:35.782+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:35.796 INFO:teuthology.orchestra.run.smithi079.stdout:1370 writesame 0x708822c thru 0x708f288 data_size 0x1(0x705d bytes) 2024-09-16T14:07:35.812 INFO:teuthology.orchestra.run.smithi079.stdout:1371 read 0x2acb724 thru 0x2acbaaa (0x387 bytes) 2024-09-16T14:07:35.813 INFO:teuthology.orchestra.run.smithi079.stdout:1372 write 0x2d22fd0 thru 0x2d2cb55 (0x9b86 bytes) 2024-09-16T14:07:35.826 INFO:teuthology.orchestra.run.smithi079.stdout:1375 read 0xda9e184 thru 0xdaa308b (0x4f08 bytes) 2024-09-16T14:07:35.826 INFO:teuthology.orchestra.run.smithi079.stdout:1376 write 0x2706f91 thru 0x270ec8c (0x7cfc bytes) 2024-09-16T14:07:35.845 INFO:teuthology.orchestra.run.smithi079.stdout:1377 punch from 0xd8a4512 to 0xd8af2ff, (0xaded bytes) 2024-09-16T14:07:35.845 INFO:teuthology.orchestra.run.smithi079.stdout:1379 write 0xaf80b28 thru 0xaf859ff (0x4ed8 bytes) 2024-09-16T14:07:35.855 INFO:teuthology.orchestra.run.smithi079.stdout:1380 writesame 0x9446cee thru 0x944d70d data_size 0x30(0x6a20 bytes) 2024-09-16T14:07:35.866 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:35.866 INFO:teuthology.orchestra.run.smithi079.stdout:1384 writesame 0x235ed3c thru 0x2360f40 data_size 0x1(0x2205 bytes) 2024-09-16T14:07:35.882 INFO:teuthology.orchestra.run.smithi079.stdout:1385 punch from 0x1f4c182 to 0x1f4f101, (0x2f7f bytes) 2024-09-16T14:07:35.883 INFO:teuthology.orchestra.run.smithi079.stdout:1386 write 0x358d19b thru 0x3590276 (0x30dc bytes) 2024-09-16T14:07:35.889 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:35.882+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:35.892 INFO:teuthology.orchestra.run.smithi079.stdout:1387 read 0xc6d929c thru 0xc6e9135 (0xfe9a bytes) 2024-09-16T14:07:35.892 INFO:teuthology.orchestra.run.smithi079.stdout:1388 punch from 0xc89e596 to 0xc8a7171, (0x8bdb bytes) 2024-09-16T14:07:35.892 INFO:teuthology.orchestra.run.smithi079.stdout:1390 write 0x580131c thru 0x580bd95 (0xaa7a bytes) 2024-09-16T14:07:35.896 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:35.896 INFO:teuthology.orchestra.run.smithi079.stdout:1392 punch from 0x5564125 to 0x556fd40, (0xbc1b bytes) 2024-09-16T14:07:35.896 INFO:teuthology.orchestra.run.smithi079.stdout:1395 read 0x3b2b27a thru 0x3b39663 (0xe3ea bytes) 2024-09-16T14:07:35.904 INFO:teuthology.orchestra.run.smithi079.stdout:1396 trunc from 0xe3f2add to 0xa05825d 2024-09-16T14:07:35.911 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:35.911 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:35.911 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:35.911 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:35.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:35.957 INFO:teuthology.orchestra.run.smithi079.stdout:1397 trunc from 0xa05825d to 0xc57f4a3 2024-09-16T14:07:35.964 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:35.964 INFO:teuthology.orchestra.run.smithi079.stdout:1399 read 0x73ffb8f thru 0x7406e61 (0x72d3 bytes) 2024-09-16T14:07:35.965 INFO:teuthology.orchestra.run.smithi079.stdout:1400 read 0xc49c654 thru 0xc49cd35 (0x6e2 bytes) 2024-09-16T14:07:35.983 INFO:teuthology.orchestra.run.smithi079.stdout:1401 writesame 0xc992890 thru 0xc998e46 data_size 0x1(0x65b7 bytes) 2024-09-16T14:07:35.983 INFO:teuthology.orchestra.run.smithi079.stdout:1402 read 0x4b31d9d thru 0x4b3adea (0x904e bytes) 2024-09-16T14:07:35.983 INFO:teuthology.orchestra.run.smithi079.stdout:1403 punch from 0x7d6dc45 to 0x7d75d56, (0x8111 bytes) 2024-09-16T14:07:35.985 INFO:teuthology.orchestra.run.smithi079.stdout:1404 read 0x3f653a4 thru 0x3f6b175 (0x5dd2 bytes) 2024-09-16T14:07:35.985 INFO:teuthology.orchestra.run.smithi079.stdout:1405 write 0x87b88f thru 0x87fb61 (0x42d3 bytes) 2024-09-16T14:07:35.990 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:35.986+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:36.006 INFO:teuthology.orchestra.run.smithi079.stdout:1406 punch from 0x9e8fb64 to 0x9e900c7, (0x563 bytes) 2024-09-16T14:07:36.006 INFO:teuthology.orchestra.run.smithi079.stdout:1407 writesame 0xb4bdf16 thru 0xb4c3102 data_size 0x1(0x51ed bytes) 2024-09-16T14:07:36.025 INFO:teuthology.orchestra.run.smithi079.stdout:1408 trunc from 0xc998e47 to 0xc662ff7 2024-09-16T14:07:36.033 INFO:teuthology.orchestra.run.smithi079.stdout:1409 writesame 0x9418e80 thru 0x9424ddf data_size 0x10(0xbf60 bytes) 2024-09-16T14:07:36.043 INFO:teuthology.orchestra.run.smithi079.stdout:1411 read 0x7cfc38d thru 0x7d0c038 (0xfcac bytes) 2024-09-16T14:07:36.044 INFO:teuthology.orchestra.run.smithi079.stdout:1412 write 0x96c893e thru 0x96c901d (0x6e0 bytes) 2024-09-16T14:07:36.047 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:36.047 INFO:teuthology.orchestra.run.smithi079.stdout:1414 write 0x6a8ad3d thru 0x6a8cda6 (0x206a bytes) 2024-09-16T14:07:36.065 INFO:teuthology.orchestra.run.smithi079.stdout:1415 trunc from 0xc662ff7 to 0x63a78e7 2024-09-16T14:07:36.071 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:36.071 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:36.071 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:36.071 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:36.071 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:36.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:36.091 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:36.086+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:36.116 INFO:teuthology.orchestra.run.smithi079.stdout:1416 punch from 0xb71337 to 0xb7a04e, (0x8d17 bytes) 2024-09-16T14:07:36.117 INFO:teuthology.orchestra.run.smithi079.stdout:1417 writesame 0x4b68b3a thru 0x4b6ae89 data_size 0x28(0x2350 bytes) 2024-09-16T14:07:36.123 INFO:teuthology.orchestra.run.smithi079.stdout:1418 punch from 0x31d41d4 to 0x31dbfaf, (0x7ddb bytes) 2024-09-16T14:07:36.125 INFO:teuthology.orchestra.run.smithi079.stdout:1419 trunc from 0x63a78e7 to 0x52a72d9 2024-09-16T14:07:36.173 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:36.173 INFO:teuthology.orchestra.run.smithi079.stdout:1422 flatten 2024-09-16T14:07:36.193 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:36.186+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:36.250 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:36.250 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:36.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 35.767693887999997, 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.095465191000000005, 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:36.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479100+0000", 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9840000000000001e-06 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479107+0000", 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7920000000000004e-06 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479120+0000", 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3195e-05 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479131+0000", 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1416999999999999e-05 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479210+0000", 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8223e-05 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479224+0000", 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.436e-05 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570355+0000", 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091131223999999997 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570668+0000", 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031290000000000002 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570990+0000", 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032147600000000001 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574367+0000", 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033769680000000002 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574482+0000", 2024-09-16T14:07:36.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115021 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574528+0000", 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6035999999999998e-05 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574560+0000", 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2595000000000002e-05 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12319 3.1d 3:bbef0c8d:::rbd_data.11364aaed44f.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.3424503730000001, 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031934152, 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12319 2024-09-16T14:07:36.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904340+0000", 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.742e-06 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904350+0000", 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6719999999999999e-06 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904372+0000", 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1787999999999999e-05 2024-09-16T14:07:36.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904381+0000", 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9099999999999994e-06 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904477+0000", 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6571999999999996e-05 2024-09-16T14:07:36.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904631+0000", 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000153311 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.912219+0000", 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075880990000000001 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:36.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936131+0000", 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023912255 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936208+0000", 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7464000000000002e-05 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936244+0000", 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5369e-05 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936273+0000", 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8969999999999999e-05 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12330 3.d 3:b20fa72b:::rbd_data.11364aaed44f.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.2679708989999998, 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.032907032000000003, 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12330 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978820+0000", 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9860000000000001e-06 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978831+0000", 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1457999999999999e-05 2024-09-16T14:07:36.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978855+0000", 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3201e-05 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978864+0000", 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6299999999999993e-06 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978954+0000", 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9420999999999996e-05 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.979080+0000", 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012617800000000001 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.986286+0000", 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072064429999999999 2024-09-16T14:07:36.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011593+0000", 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025306443000000001 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011654+0000", 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0764999999999998e-05 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011696+0000", 2024-09-16T14:07:36.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2330999999999998e-05 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011725+0000", 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9176e-05 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15971 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 851968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178362+0000", 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068427209000000003, 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021997244999999999, 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15971 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178362+0000", 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178362+0000", 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178362+0000", 2024-09-16T14:07:36.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3000000000000002e-07 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178363+0000", 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3239999999999999e-06 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178367+0000", 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178369+0000", 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200267+0000", 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021897349 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200312+0000", 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5181e-05 2024-09-16T14:07:36.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200359+0000", 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7042000000000001e-05 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15973 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178382+0000", 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068407351000000005, 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022094526999999999, 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15973 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.277 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178382+0000", 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178382+0000", 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178382+0000", 2024-09-16T14:07:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3300000000000003e-07 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178383+0000", 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1880000000000001e-06 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178387+0000", 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178389+0000", 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4269999999999998e-06 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200375+0000", 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021986061000000001 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200429+0000", 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3464000000000002e-05 2024-09-16T14:07:36.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200476+0000", 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7478000000000002e-05 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15974 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068397487000000007, 2024-09-16T14:07:36.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022185745, 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15974 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178392+0000", 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178393+0000", 2024-09-16T14:07:36.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.308e-06 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178396+0000", 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4209999999999999e-06 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178399+0000", 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3120000000000001e-06 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200491+0000", 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022092166999999999 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200529+0000", 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8380999999999997e-05 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200577+0000", 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7868000000000003e-05 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.285 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15976 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068378116000000003, 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022266966999999999, 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15976 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178412+0000", 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178416+0000", 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4489999999999999e-06 2024-09-16T14:07:36.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178418+0000", 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2900000000000001e-06 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200596+0000", 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022178192999999999 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200633+0000", 2024-09-16T14:07:36.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6792999999999998e-05 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200678+0000", 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4743999999999997e-05 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15977 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:36.290 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068368351999999993, 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022367227999999999, 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15977 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.84e-07 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178422+0000", 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.257e-06 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178426+0000", 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178429+0000", 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4200000000000001e-06 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200694+0000", 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022265367000000001 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200735+0000", 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0908000000000002e-05 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200788+0000", 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2787999999999997e-05 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15978 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068357707000000004, 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022476471000000001, 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.295 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15978 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178432+0000", 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0800000000000001e-07 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178435+0000", 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7249999999999999e-06 2024-09-16T14:07:36.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178439+0000", 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.507e-06 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178441+0000", 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5579999999999999e-06 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200803+0000", 2024-09-16T14:07:36.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022361322999999999 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200841+0000", 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8353999999999999e-05 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200908+0000", 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6696e-05 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15980 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068333570999999996, 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022615139999999999, 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.300 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15980 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178456+0000", 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8299999999999998e-07 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178457+0000", 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.271e-06 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178460+0000", 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.455e-06 2024-09-16T14:07:36.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178463+0000", 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.306e-06 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200933+0000", 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022470054999999999 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201000+0000", 2024-09-16T14:07:36.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7117000000000001e-05 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201071+0000", 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0653000000000003e-05 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15982 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:36.304 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068314056999999997, 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022793758000000001, 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15982 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0499999999999999e-07 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178479+0000", 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.433e-06 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178482+0000", 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.63e-06 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178485+0000", 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4789999999999999e-06 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201098+0000", 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022612818 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201185+0000", 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7886999999999994e-05 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201269+0000", 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3206000000000003e-05 2024-09-16T14:07:36.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15983 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068301728000000006, 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022918754, 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15983 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178488+0000", 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178489+0000", 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.373e-06 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178492+0000", 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5609999999999999e-06 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178495+0000", 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3499999999999999e-06 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201297+0000", 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022801920999999999 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201345+0000", 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8643000000000002e-05 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201406+0000", 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0571e-05 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15984 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068286436000000006, 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023018235000000001, 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15984 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:36.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178504+0000", 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.37e-06 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178508+0000", 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4879999999999999e-06 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178510+0000", 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3779999999999999e-06 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201424+0000", 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022913830999999999 2024-09-16T14:07:36.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201461+0000", 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7002000000000002e-05 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201521+0000", 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9620999999999997e-05 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15986 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068243861000000003, 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023085504, 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15986 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178546+0000", 2024-09-16T14:07:36.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7199999999999999e-07 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178548+0000", 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.097e-06 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178554+0000", 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2820000000000002e-06 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178558+0000", 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.878e-06 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201540+0000", 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022982318000000002 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201575+0000", 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5138999999999997e-05 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201631+0000", 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5217999999999999e-05 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15988 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068209729999999996, 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023229864999999999, 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.323 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15988 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178580+0000", 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1300000000000001e-07 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178582+0000", 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178589+0000", 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178594+0000", 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1459999999999998e-06 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201654+0000", 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023060664000000002 2024-09-16T14:07:36.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201725+0000", 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0451999999999996e-05 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201809+0000", 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4395999999999996e-05 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15989 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068190773999999996, 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023391981999999999, 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15989 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.328 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178599+0000", 2024-09-16T14:07:36.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178601+0000", 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178608+0000", 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9789999999999996e-06 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178612+0000", 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0890000000000002e-06 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201835+0000", 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023222702000000001 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201905+0000", 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0425000000000004e-05 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201990+0000", 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5062999999999995e-05 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15990 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068172601999999999, 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023564649, 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15990 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178617+0000", 2024-09-16T14:07:36.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.63e-07 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178619+0000", 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.097e-06 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178626+0000", 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6320000000000002e-06 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178629+0000", 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8480000000000003e-06 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202017+0000", 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023387811000000001 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202111+0000", 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4055999999999995e-05 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202181+0000", 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9741999999999997e-05 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15992 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068132363000000001, 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02370862, 2024-09-16T14:07:36.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15992 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6600000000000002e-07 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178659+0000", 2024-09-16T14:07:36.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3379999999999999e-06 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178666+0000", 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7050000000000002e-06 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178670+0000", 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6789999999999998e-06 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202206+0000", 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023536148999999999 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202248+0000", 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2539e-05 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202365+0000", 2024-09-16T14:07:36.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011674400000000001 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15993 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068115049999999996, 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024088607000000001, 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15993 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:36.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999999e-07 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178677+0000", 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2180000000000002e-06 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178684+0000", 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0770000000000002e-06 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178694+0000", 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0241000000000001e-05 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202580+0000", 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023885937999999999 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202667+0000", 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7425999999999999e-05 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202763+0000", 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5235000000000006e-05 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15995 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.068030206999999995, 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024187608999999999, 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15995 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2800000000000003e-07 2024-09-16T14:07:36.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178761+0000", 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6810000000000001e-06 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178765+0000", 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4259999999999996e-06 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178768+0000", 2024-09-16T14:07:36.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6740000000000001e-06 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202791+0000", 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024023224999999999 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202865+0000", 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4206000000000001e-05 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:36.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202946+0000", 2024-09-16T14:07:36.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1069000000000006e-05 2024-09-16T14:07:36.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.351 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:36.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:36.353 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:36.346+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:36.386 INFO:teuthology.orchestra.run.smithi079.stdout:1423 trunc from 0x52a72d9 to 0xe21e90e 2024-09-16T14:07:36.395 INFO:teuthology.orchestra.run.smithi079.stdout:1424 write 0xb6d9600 thru 0xb6e7694 (0xe095 bytes) 2024-09-16T14:07:36.414 INFO:teuthology.orchestra.run.smithi079.stdout:1426 read 0x45aecc thru 0x46a88f (0xf9c4 bytes) 2024-09-16T14:07:36.414 INFO:teuthology.orchestra.run.smithi079.stdout:1427 punch from 0x8fde4f to 0x906009, (0x81ba bytes) 2024-09-16T14:07:36.423 INFO:teuthology.orchestra.run.smithi079.stdout:1430 trunc from 0xe21e90e to 0x890e435 2024-09-16T14:07:36.440 INFO:teuthology.orchestra.run.smithi079.stdout:1431 writesame 0x31bcfdc thru 0x31c254c data_size 0x3(0x5571 bytes) 2024-09-16T14:07:36.449 INFO:teuthology.orchestra.run.smithi079.stdout:1432 read 0x83187fd thru 0x8325f5c (0xd760 bytes) 2024-09-16T14:07:36.449 INFO:teuthology.orchestra.run.smithi079.stdout:1433 read 0x6a565d9 thru 0x6a62b70 (0xc598 bytes) 2024-09-16T14:07:36.449 INFO:teuthology.orchestra.run.smithi079.stdout:1435 punch from 0xe7ad to 0x1aca7, (0xc4fa bytes) 2024-09-16T14:07:36.450 INFO:teuthology.orchestra.run.smithi079.stdout:1436 read 0x2bf5df1 thru 0x2bfda45 (0x7c55 bytes) 2024-09-16T14:07:36.451 INFO:teuthology.orchestra.run.smithi079.stdout:1437 read 0x25619b7 thru 0x2563828 (0x1e72 bytes) 2024-09-16T14:07:36.451 INFO:teuthology.orchestra.run.smithi079.stdout:1440 read 0x4ab8501 thru 0x4ac1ccd (0x97cd bytes) 2024-09-16T14:07:36.451 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:36.451 INFO:teuthology.orchestra.run.smithi079.stdout:1442 punch from 0x382ba75 to 0x3838444, (0xc9cf bytes) 2024-09-16T14:07:36.451 INFO:teuthology.orchestra.run.smithi079.stdout:1445 read 0x5ae90d4 thru 0x5aea494 (0x13c1 bytes) 2024-09-16T14:07:36.451 INFO:teuthology.orchestra.run.smithi079.stdout:1446 read 0x85e891b thru 0x85f2b5b (0xa241 bytes) 2024-09-16T14:07:36.452 INFO:teuthology.orchestra.run.smithi079.stdout:1447 read 0x27ba559 thru 0x27c37b6 (0x925e bytes) 2024-09-16T14:07:36.453 INFO:teuthology.orchestra.run.smithi079.stdout:truncating to largest ever: 0xed64c54 2024-09-16T14:07:36.453 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:36.446+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:36.456 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:36.456 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:36.456 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:36.457 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:36.462 INFO:teuthology.orchestra.run.smithi079.stdout:1448 trunc from 0x890e435 to 0xed64c54 2024-09-16T14:07:36.466 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:36.469 INFO:teuthology.orchestra.run.smithi079.stdout:1449 read 0xa25d165 thru 0xa2654d5 (0x8371 bytes) 2024-09-16T14:07:36.470 INFO:teuthology.orchestra.run.smithi079.stdout:1450 read 0xbb6fece thru 0xbb7af69 (0xb09c bytes) 2024-09-16T14:07:36.470 INFO:teuthology.orchestra.run.smithi079.stdout:1451 write 0x542495b thru 0x54283b9 (0x3a5f bytes) 2024-09-16T14:07:36.473 INFO:teuthology.orchestra.run.smithi079.stdout:1452 read 0x3a11a42 thru 0x3a17e21 (0x63e0 bytes) 2024-09-16T14:07:36.474 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:36.474 INFO:teuthology.orchestra.run.smithi079.stdout:1456 write 0x496fc76 thru 0x4976243 (0x65ce bytes) 2024-09-16T14:07:36.488 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:36.488 INFO:teuthology.orchestra.run.smithi079.stdout:1459 read 0x2e65097 thru 0x2e6f419 (0xa383 bytes) 2024-09-16T14:07:36.488 INFO:teuthology.orchestra.run.smithi079.stdout:1460 write 0xcd6bfab thru 0xcd71b80 (0x5bd6 bytes) 2024-09-16T14:07:36.498 INFO:teuthology.orchestra.run.smithi079.stdout:1461 read 0x7608ebd thru 0x760a873 (0x19b7 bytes) 2024-09-16T14:07:36.498 INFO:teuthology.orchestra.run.smithi079.stdout:1462 read 0x6da6ab thru 0x6e815a (0xdab0 bytes) 2024-09-16T14:07:36.498 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:36.498 INFO:teuthology.orchestra.run.smithi079.stdout:1465 writesame 0x5765a43 thru 0x5771a94 data_size 0x67(0xc052 bytes) 2024-09-16T14:07:36.512 INFO:teuthology.orchestra.run.smithi079.stdout:1466 read 0xa7c3cd3 thru 0xa7ce7bf (0xaaed bytes) 2024-09-16T14:07:36.512 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:36.512 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:36.513 INFO:teuthology.orchestra.run.smithi079.stdout:1473 punch from 0xb19ce3b to 0xb1a5fc5, (0x918a bytes) 2024-09-16T14:07:36.513 INFO:teuthology.orchestra.run.smithi079.stdout:1476 write 0x6b0e6d9 thru 0x6b0f930 (0x1258 bytes) 2024-09-16T14:07:36.517 INFO:teuthology.orchestra.run.smithi079.stdout:1477 writesame 0x2e2ab7a thru 0x2e32fd7 data_size 0x1(0x845e bytes) 2024-09-16T14:07:36.542 INFO:teuthology.orchestra.run.smithi079.stdout:1478 trunc from 0xed64c54 to 0x5c7ec8d 2024-09-16T14:07:36.553 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:36.546+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:36.561 INFO:teuthology.orchestra.run.smithi079.stdout:1479 write 0x22de84f thru 0x22e28b3 (0x4065 bytes) 2024-09-16T14:07:36.567 INFO:teuthology.orchestra.run.smithi079.stdout:1481 punch from 0x3dba4b1 to 0x3dc43b5, (0x9f04 bytes) 2024-09-16T14:07:36.567 INFO:teuthology.orchestra.run.smithi079.stdout:1483 punch from 0x3a1e099 to 0x3a20717, (0x267e bytes) 2024-09-16T14:07:36.568 INFO:teuthology.orchestra.run.smithi079.stdout:1484 write 0x37afd73 thru 0x37afed7 (0x165 bytes) 2024-09-16T14:07:36.573 INFO:teuthology.orchestra.run.smithi079.stdout:1485 punch from 0x1a84d53 to 0x1a8e805, (0x9ab2 bytes) 2024-09-16T14:07:36.573 INFO:teuthology.orchestra.run.smithi079.stdout:1486 read 0x47b2aca thru 0x47bdc38 (0xb16f bytes) 2024-09-16T14:07:36.573 INFO:teuthology.orchestra.run.smithi079.stdout:1487 punch from 0x3aed2e0 to 0x3afb232, (0xdf52 bytes) 2024-09-16T14:07:36.573 INFO:teuthology.orchestra.run.smithi079.stdout:1488 writesame 0x9cb387 thru 0x9d1e24 data_size 0x3(0x6a9e bytes) 2024-09-16T14:07:36.573 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:36.574 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:36.583 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:36.601 INFO:teuthology.orchestra.run.smithi079.stdout:1489 write 0xa00b24c thru 0xa010c54 (0x5a09 bytes) 2024-09-16T14:07:36.606 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:07:36.606 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:07:36.606 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T14:07:36.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T14:07:36.627 INFO:teuthology.orchestra.run.smithi079.stdout:1490 trunc from 0xa010c55 to 0x3c7e994 2024-09-16T14:07:36.655 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:36.650+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:36.658 INFO:teuthology.orchestra.run.smithi079.stdout:1491 trunc from 0x3c7e994 to 0x31bb6aa 2024-09-16T14:07:36.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:36.686 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:918 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.950862240999999, 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015774501, 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 918 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730965+0000", 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:07:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730967+0000", 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730971+0000", 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5080000000000001e-06 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730973+0000", 2024-09-16T14:07:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.091e-06 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.739992+0000", 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090191430000000003 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740162+0000", 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016968899999999999 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740834+0000", 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067256100000000001 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.744120+0000", 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032859370000000001 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746643+0000", 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025232050000000002 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746700+0000", 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6251000000000001e-05 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746739+0000", 2024-09-16T14:07:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9201e-05 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1141 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.344987434, 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014042171000000001, 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1141 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336840+0000", 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2600000000000002e-07 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336844+0000", 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7419999999999999e-06 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336850+0000", 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336855+0000", 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4939999999999997e-06 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347139+0000", 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010284331000000001 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347252+0000", 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011294300000000001 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347714+0000", 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046169700000000001 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350720+0000", 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030062359999999998 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350774+0000", 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3415999999999998e-05 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350837+0000", 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3151999999999994e-05 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350881+0000", 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4591e-05 2024-09-16T14:07:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1142 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.344967184000001, 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016069268000000001, 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1142 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336860+0000", 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336866+0000", 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5600000000000001e-06 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336872+0000", 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9410000000000001e-06 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336877+0000", 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.172e-06 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348737+0000", 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011860239999999999 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348829+0000", 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1657999999999996e-05 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349140+0000", 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031145499999999998 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352798+0000", 2024-09-16T14:07:36.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036581550000000002 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352815+0000", 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6883000000000001e-05 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352876+0000", 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0871e-05 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352929+0000", 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2781000000000002e-05 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1143 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.344945785, 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018939362000000001, 2024-09-16T14:07:36.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1143 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.44e-07 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336884+0000", 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336891+0000", 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336895+0000", 2024-09-16T14:07:36.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0820000000000001e-06 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349584+0000", 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012689283000000001 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349708+0000", 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123793 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350151+0000", 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044259900000000001 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352993+0000", 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028426979999999998 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355720+0000", 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002726642 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355762+0000", 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2116000000000001e-05 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355820+0000", 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8118999999999997e-05 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.344927679000001, 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:07:36.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:07:36.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.344905281999999, 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:36.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:07:36.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.344298570999999, 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:07:36.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:07:36.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.344224243999999, 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.731 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:07:36.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:07:36.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1150 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.336874324, 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017570243999999999, 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1150 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344954+0000", 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4959999999999999e-06 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344964+0000", 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8719999999999997e-06 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344980+0000", 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6622000000000001e-05 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344988+0000", 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6879999999999999e-06 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354864+0000", 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098759160000000002 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354938+0000", 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4179999999999998e-05 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355250+0000", 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000311973 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.361925+0000", 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066749859999999999 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362285+0000", 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036007999999999998 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362462+0000", 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017646200000000001 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362523+0000", 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1151 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.336385574000001, 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017278714000000001, 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1151 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345442+0000", 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.42e-06 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345450+0000", 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2420000000000003e-06 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345461+0000", 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1395e-05 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345467+0000", 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3600000000000001e-06 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355848+0000", 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010380162 2024-09-16T14:07:36.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355940+0000", 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2629999999999999e-05 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356352+0000", 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041188100000000001 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362037+0000", 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005684911 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362545+0000", 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050792899999999995 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362636+0000", 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1479999999999998e-05 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362720+0000", 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3304000000000001e-05 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1154 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.334004349000001, 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015133097999999999, 2024-09-16T14:07:36.749 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1154 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347823+0000", 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347832+0000", 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347854+0000", 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2086000000000002e-05 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347864+0000", 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356827+0000", 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089628060000000002 2024-09-16T14:07:36.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356897+0000", 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0494000000000006e-05 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357171+0000", 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027356199999999998 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362741+0000", 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055696920000000002 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362867+0000", 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012610799999999999 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362922+0000", 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4911000000000003e-05 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362955+0000", 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3810999999999999e-05 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1155 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.333717878000002, 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014975097999999999, 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1155 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:36.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348110+0000", 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6700000000000002e-07 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348114+0000", 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2560000000000004e-06 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348129+0000", 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4725e-05 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348134+0000", 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.152e-06 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357588+0000", 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094536949999999998 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357666+0000", 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8664000000000004e-05 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357994+0000", 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032747700000000001 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362762+0000", 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047681829999999996 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363017+0000", 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025504500000000002 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363056+0000", 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9357000000000003e-05 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363084+0000", 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7577e-05 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1161 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.330745275999998, 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017258941999999999, 2024-09-16T14:07:36.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1161 2024-09-16T14:07:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351083+0000", 2024-09-16T14:07:36.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2270000000000001e-06 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351089+0000", 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.178e-06 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351108+0000", 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9457e-05 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351116+0000", 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6499999999999996e-06 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360527+0000", 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094112959999999996 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360645+0000", 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011791299999999999 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.361269+0000", 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062395199999999999 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365240+0000", 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039707470000000002 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368249+0000", 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030092000000000001 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368302+0000", 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3146999999999998e-05 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368340+0000", 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8175e-05 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1162 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.329017532000002, 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017877857, 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.771 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1162 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352810+0000", 2024-09-16T14:07:36.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.234e-06 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352818+0000", 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352831+0000", 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2839e-05 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352838+0000", 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1790000000000002e-06 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363259+0000", 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010421616 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363370+0000", 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111134 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363713+0000", 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034221700000000002 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.367148+0000", 2024-09-16T14:07:36.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034350029999999998 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370616+0000", 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034687419999999999 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370660+0000", 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3365000000000002e-05 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370687+0000", 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7121999999999999e-05 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1164 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.327941506999998, 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016868919, 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.778 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1164 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353886+0000", 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2130000000000001e-06 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353892+0000", 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1299999999999998e-06 2024-09-16T14:07:36.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353903+0000", 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0821e-05 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353909+0000", 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2909999999999998e-06 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364201+0000", 2024-09-16T14:07:36.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010292318 2024-09-16T14:07:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364295+0000", 2024-09-16T14:07:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3667999999999998e-05 2024-09-16T14:07:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364657+0000", 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036279999999999998 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.367209+0000", 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025513250000000001 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370697+0000", 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034881600000000001 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370729+0000", 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1977999999999998e-05 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370754+0000", 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5215000000000002e-05 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1166 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.325693922999999, 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014686309999999999, 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1166 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:36.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9400000000000004e-07 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356137+0000", 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.473e-06 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356161+0000", 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3968000000000002e-05 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356167+0000", 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5840000000000002e-06 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365282+0000", 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091153269999999995 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365369+0000", 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7064000000000005e-05 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365709+0000", 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034009900000000001 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368388+0000", 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002679393 2024-09-16T14:07:36.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370762+0000", 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023732319999999999 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370793+0000", 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1550000000000001e-05 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370819+0000", 2024-09-16T14:07:36.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5826e-05 2024-09-16T14:07:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1325 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:36.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.962085756, 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014200757, 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1325 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719742+0000", 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3459999999999999e-06 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719750+0000", 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6569999999999997e-06 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719777+0000", 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7370000000000001e-05 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719785+0000", 2024-09-16T14:07:36.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2190000000000004e-06 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728088+0000", 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083028860000000006 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728229+0000", 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014092799999999999 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:36.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728704+0000", 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047427599999999998 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.729579+0000", 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00087516299999999998 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733865+0000", 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042860570000000002 2024-09-16T14:07:36.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733907+0000", 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789999999999998e-05 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733942+0000", 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5064999999999997e-05 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.3732994519999995, 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:07:36.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7453408709999998, 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.802 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:07:36.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:36.808 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.2151804049999999, 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:36.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:07:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:07:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:36.814 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:36.816 INFO:teuthology.orchestra.run.smithi079.stdout:1492 writesame 0x718d204 thru 0x719a408 data_size 0x1(0xd205 bytes) 2024-09-16T14:07:36.816 INFO:teuthology.orchestra.run.smithi079.stdout:1493 read 0x5d531c8 thru 0x5d61f7b (0xedb4 bytes) 2024-09-16T14:07:36.816 INFO:teuthology.orchestra.run.smithi079.stdout:1494 read 0x583b1e9 thru 0x583f8d6 (0x46ee bytes) 2024-09-16T14:07:36.816 INFO:teuthology.orchestra.run.smithi079.stdout:1496 read 0x1fb1e04 thru 0x1fbcbf9 (0xadf6 bytes) 2024-09-16T14:07:36.817 INFO:teuthology.orchestra.run.smithi079.stdout:1497 clone 11 order 18 su 1024 sc 11 2024-09-16T14:07:36.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:36.818 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:36.810+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:36.818 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:07:36.819 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:36.821 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:36.838 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:36.857 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:36.876 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:36.909 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:36.916 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:36.910+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:36.927 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-16T14:07:36.930 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:36.930 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:36.939 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:36.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:36.959 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:36.959 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:36.959 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 46, 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "a22b38a6-94db-470d-bf35-45d7605f3bb0", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-16T14:06:40.573726+0000", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-16T14:07:36.714625+0000", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-16T14:06:51.260361+0000", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-16T14:06:42.748450+0000", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-16T14:07:36.960 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 5, 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 3, 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "jewel", 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "quincy", 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-16T14:07:36.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:52.822826+0000", 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:07:36.962 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.963 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "18", 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:07:36.964 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:07:36.965 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:07:36.970 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:07:36.970 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:07:36.970 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:07:36.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.970 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:07:36.971 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:07:36.971 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:07:36.971 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:07:36.971 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:07:36.971 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:07:36.972 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:07:36.972 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:07:36.972 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:07:36.972 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:07:36.972 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:07:36.972 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-16T14:07:36.973 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-16T14:07:36.973 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-16T14:07:36.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.973 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:07:36.973 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-16T14:07:36.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:58.526935+0000", 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:07:36.974 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:07:36.975 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:07:36.975 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:07:36.975 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:07:36.975 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:07:36.975 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:07:36.975 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:07:36.976 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "22", 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 22, 2024-09-16T14:07:36.977 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:07:36.978 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:07:36.979 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:07:36.980 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "pool_client.0", 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:07:08.375919+0000", 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:07:36.981 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:07:36.982 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "46", 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:07:36.983 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 14, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 46, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:07:36.984 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:07:36.985 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.986 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a060b161-fd5a-4cfb-bac8-f2acc32d7955", 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:36.987 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 34, 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:36.988 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.990 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:36.990 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-16T14:07:36.990 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:36.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.990 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:36.990 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.992 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/22196", 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/22196", 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6824/22196", 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/22196", 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.993 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "f5ba4b39-ceb8-46b3-938e-57249ed84b87", 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 34, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:36.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:36.995 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.996 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:36.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.998 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.998 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:36.998 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:36.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.998 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:36.998 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-16T14:07:36.998 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/22195", 2024-09-16T14:07:36.999 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/22195", 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/22195", 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/22195", 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "8b657f92-66c9-46a3-a31f-0bd0be414e4e", 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:37.000 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 34, 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-16T14:07:37.001 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-16T14:07:37.002 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.003 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.005 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6825/22197", 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6827/22197", 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6831/22197", 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6829/22197", 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:07:37.006 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "15937bd7-8964-4140-9f86-b3093c318d1e", 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.008 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:37.009 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.010 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/22193", 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/22193", 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/22193", 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/22193", 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:37.012 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "714baf80-ade0-4e88-ad4e-5171add1fc6a", 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.242828369140625, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 14, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:37.013 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6808", 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6809", 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6810", 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6811", 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.016 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6814", 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6815", 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6812", 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6813", 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:07:37.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6809/22362", 2024-09-16T14:07:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6811/22362", 2024-09-16T14:07:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6815/22362", 2024-09-16T14:07:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6813/22362", 2024-09-16T14:07:37.019 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e5a21b84-81ab-45da-a081-3b504f5c4f3d", 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:37.020 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6800", 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6801", 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6802", 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6803", 2024-09-16T14:07:37.023 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6806", 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.024 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6807", 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6804", 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.025 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6805", 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6801/22363", 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6803/22363", 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6807/22363", 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6805/22363", 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:37.026 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:37.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:07:37.027 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a009857f-f798-4154-9ff4-f524f60b6c61", 2024-09-16T14:07:37.027 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:37.027 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:37.028 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:37.028 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:37.028 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:37.028 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:37.028 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:37.029 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 34, 2024-09-16T14:07:37.029 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:37.029 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:37.029 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:37.029 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.030 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6824", 2024-09-16T14:07:37.030 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:37.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.031 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6825", 2024-09-16T14:07:37.031 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:37.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.031 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.031 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:37.032 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.032 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.032 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6826", 2024-09-16T14:07:37.032 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:37.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.033 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.033 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6827", 2024-09-16T14:07:37.033 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:37.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.034 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.034 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:37.034 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.034 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.035 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6830", 2024-09-16T14:07:37.035 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:37.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.035 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6831", 2024-09-16T14:07:37.036 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:37.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.036 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:37.037 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.037 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6828", 2024-09-16T14:07:37.037 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:37.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.038 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.038 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6829", 2024-09-16T14:07:37.038 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:07:37.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.038 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.039 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6825/22364", 2024-09-16T14:07:37.039 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6827/22364", 2024-09-16T14:07:37.039 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6831/22364", 2024-09-16T14:07:37.039 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6829/22364", 2024-09-16T14:07:37.039 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:37.039 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:37.040 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:37.040 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.040 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:07:37.040 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e3506002-ba8b-4251-8ef1-136334574a6d", 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 34, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:07:37.041 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6816", 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6817", 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.042 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6818", 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.043 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6819", 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6822", 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6823", 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6820", 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:37.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6821", 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6817/22365", 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6819/22365", 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6823/22365", 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6821/22365", 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:07:37.046 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:37.047 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.104631+0000", 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987403+0000", 2024-09-16T14:07:37.048 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.018134+0000", 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.049 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987645+0000", 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.050 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:07:37.051 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:37.051 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:37.051 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:37.051 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:37.051 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:37.051 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.721507+0000", 2024-09-16T14:07:37.051 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.869796+0000", 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:37.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.206576+0000", 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:07:37.053 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.038098+0000", 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-16T14:07:37.054 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-16T14:07:37.055 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:37.056 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:37.057 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:37.058 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:37.058 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:37.058 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:37.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:37.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:37.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:37.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:37.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:37.060 INFO:tasks.thrashosds.thrasher:j is {'epoch': 46, 'fsid': 'a22b38a6-94db-470d-bf35-45d7605f3bb0', 'created': '2024-09-16T14:06:40.573726+0000', 'modified': '2024-09-16T14:07:36.714625+0000', 'last_up_change': '2024-09-16T14:06:51.260361+0000', 'last_in_change': '2024-09-16T14:06:42.748450+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T14:06:52.822826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T14:06:58.526935+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T14:07:08.375919+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 46, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': 'a060b161-fd5a-4cfb-bac8-f2acc32d7955', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 22196}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 22196}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6824', 'nonce': 22196}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 22196}]}, 'public_addr': '172.21.15.79:6817/22196', 'cluster_addr': '172.21.15.79:6819/22196', 'heartbeat_back_addr': '172.21.15.79:6824/22196', 'heartbeat_front_addr': '172.21.15.79:6821/22196', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f5ba4b39-ceb8-46b3-938e-57249ed84b87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 22195}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 22195}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 22195}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 22195}]}, 'public_addr': '172.21.15.79:6809/22195', 'cluster_addr': '172.21.15.79:6811/22195', 'heartbeat_back_addr': '172.21.15.79:6815/22195', 'heartbeat_front_addr': '172.21.15.79:6813/22195', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8b657f92-66c9-46a3-a31f-0bd0be414e4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6823', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 22197}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 22197}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 22197}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 22197}]}, 'public_addr': '172.21.15.79:6825/22197', 'cluster_addr': '172.21.15.79:6827/22197', 'heartbeat_back_addr': '172.21.15.79:6831/22197', 'heartbeat_front_addr': '172.21.15.79:6829/22197', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '15937bd7-8964-4140-9f86-b3093c318d1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 22193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 22193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 22193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 22193}]}, 'public_addr': '172.21.15.79:6801/22193', 'cluster_addr': '172.21.15.79:6803/22193', 'heartbeat_back_addr': '172.21.15.79:6807/22193', 'heartbeat_front_addr': '172.21.15.79:6805/22193', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '714baf80-ade0-4e88-ad4e-5171add1fc6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.242828369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 22362}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 22362}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 22362}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 22362}]}, 'public_addr': '172.21.15.164:6809/22362', 'cluster_addr': '172.21.15.164:6811/22362', 'heartbeat_back_addr': '172.21.15.164:6815/22362', 'heartbeat_front_addr': '172.21.15.164:6813/22362', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e5a21b84-81ab-45da-a081-3b504f5c4f3d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 22363}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 22363}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 22363}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 22363}]}, 'public_addr': '172.21.15.164:6801/22363', 'cluster_addr': '172.21.15.164:6803/22363', 'heartbeat_back_addr': '172.21.15.164:6807/22363', 'heartbeat_front_addr': '172.21.15.164:6805/22363', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a009857f-f798-4154-9ff4-f524f60b6c61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 22364}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 22364}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 22364}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 22364}]}, 'public_addr': '172.21.15.164:6825/22364', 'cluster_addr': '172.21.15.164:6827/22364', 'heartbeat_back_addr': '172.21.15.164:6831/22364', 'heartbeat_front_addr': '172.21.15.164:6829/22364', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e3506002-ba8b-4251-8ef1-136334574a6d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 22365}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 22365}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 22365}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 22365}]}, 'public_addr': '172.21.15.164:6817/22365', 'cluster_addr': '172.21.15.164:6819/22365', 'heartbeat_back_addr': '172.21.15.164:6823/22365', 'heartbeat_front_addr': '172.21.15.164:6821/22365', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.104631+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987403+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.018134+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987645+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.721507+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.869796+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.206576+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.038098+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T14:07:37.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:07:37.061 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:37.050+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:37.158 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:37.150+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:37.179 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:37.179 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:37.189 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:37.259 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:37.254+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:37.351 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:37.352 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:37.360 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:37.354+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:37.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:37.418 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:37.422 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:07:37.432 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-09-16T14:07:36.771095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":271415438,"num_objects":600,"num_object_clones":169,"num_object_copies":1200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":600,"num_whiteouts":187,"num_read":22964,"num_read_kb":55923,"num_write":5033,"num_write_kb":65820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3931,"ondisk_log_size":3931,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2434508,"kb_used_data":14048,"kb_used_omap":13,"kb_used_meta":2420338,"kb_avail":752540212,"statfs":{"total":773094113280,"available":770601177088,"internally_reserved":0,"allocated":14385152,"data_stored":72723352,"data_compressed":3125768,"data_compressed_allocated":7208960,"data_compressed_original":68912544,"omap_allocated":13632,"internal_metadata":2478426816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":783384,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":12,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.077757"},"pg_stats":[{"pgid":"3.1f","version":"41'263","reported_seq":897,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.479965+0000","last_change":"2024-09-16T14:07:11.338271+0000","last_active":"2024-09-16T14:07:29.479965+0000","last_peered":"2024-09-16T14:07:29.479965+0000","last_clean":"2024-09-16T14:07:29.479965+0000","last_became_active":"2024-09-16T14:07:09.540900+0000","last_became_peered":"2024-09-16T14:07:09.540900+0000","last_unstale":"2024-09-16T14:07:29.479965+0000","last_undegraded":"2024-09-16T14:07:29.479965+0000","last_fullsized":"2024-09-16T14:07:29.479965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:59.820596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233770000000001,"stat_sum":{"num_bytes":13238264,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2466,"num_read_kb":3485,"num_write":415,"num_write_kb":1726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"41'46","reported_seq":318,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.630464+0000","last_change":"2024-09-16T14:07:11.337654+0000","last_active":"2024-09-16T14:07:29.630464+0000","last_peered":"2024-09-16T14:07:29.630464+0000","last_clean":"2024-09-16T14:07:29.630464+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:07:29.630464+0000","last_undegraded":"2024-09-16T14:07:29.630464+0000","last_fullsized":"2024-09-16T14:07:29.630464+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060911299999999999,"stat_sum":{"num_bytes":7570510,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":270,"num_read_kb":1056,"num_write":28,"num_write_kb":245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"42'68","reported_seq":201,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.650304+0000","last_change":"2024-09-16T14:07:11.546356+0000","last_active":"2024-09-16T14:07:31.650304+0000","last_peered":"2024-09-16T14:07:31.650304+0000","last_clean":"2024-09-16T14:07:31.650304+0000","last_became_active":"2024-09-16T14:07:09.552487+0000","last_became_peered":"2024-09-16T14:07:09.552487+0000","last_unstale":"2024-09-16T14:07:31.650304+0000","last_undegraded":"2024-09-16T14:07:31.650304+0000","last_fullsized":"2024-09-16T14:07:31.650304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:53:07.333781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072576699999999999,"stat_sum":{"num_bytes":9249960,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":129,"num_read_kb":411,"num_write":61,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"41'72","reported_seq":353,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.448074+0000","last_change":"2024-09-16T14:07:11.340416+0000","last_active":"2024-09-16T14:07:29.448074+0000","last_peered":"2024-09-16T14:07:29.448074+0000","last_clean":"2024-09-16T14:07:29.448074+0000","last_became_active":"2024-09-16T14:07:09.551657+0000","last_became_peered":"2024-09-16T14:07:09.551657+0000","last_unstale":"2024-09-16T14:07:29.448074+0000","last_undegraded":"2024-09-16T14:07:29.448074+0000","last_fullsized":"2024-09-16T14:07:29.448074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:49.635426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064309800000000004,"stat_sum":{"num_bytes":5519232,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":282,"num_read_kb":579,"num_write":62,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"41'141","reported_seq":422,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.464704+0000","last_change":"2024-09-16T14:07:22.350217+0000","last_active":"2024-09-16T14:07:29.464704+0000","last_peered":"2024-09-16T14:07:29.464704+0000","last_clean":"2024-09-16T14:07:29.464704+0000","last_became_active":"2024-09-16T14:07:22.349546+0000","last_became_peered":"2024-09-16T14:07:22.349546+0000","last_unstale":"2024-09-16T14:07:29.464704+0000","last_undegraded":"2024-09-16T14:07:29.464704+0000","last_fullsized":"2024-09-16T14:07:29.464704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:30.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021275310000000002,"stat_sum":{"num_bytes":7180835,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":682,"num_read_kb":674,"num_write":208,"num_write_kb":886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"42'153","reported_seq":770,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.703091+0000","last_change":"2024-09-16T14:07:11.340816+0000","last_active":"2024-09-16T14:07:31.703091+0000","last_peered":"2024-09-16T14:07:31.703091+0000","last_clean":"2024-09-16T14:07:31.703091+0000","last_became_active":"2024-09-16T14:07:09.540237+0000","last_became_peered":"2024-09-16T14:07:09.540237+0000","last_unstale":"2024-09-16T14:07:31.703091+0000","last_undegraded":"2024-09-16T14:07:31.703091+0000","last_fullsized":"2024-09-16T14:07:31.703091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:38.147579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099540499999999999,"stat_sum":{"num_bytes":14486741,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":986,"num_read_kb":6205,"num_write":194,"num_write_kb":17198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"41'50","reported_seq":259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.472764+0000","last_change":"2024-09-16T14:07:22.346713+0000","last_active":"2024-09-16T14:07:29.472764+0000","last_peered":"2024-09-16T14:07:29.472764+0000","last_clean":"2024-09-16T14:07:29.472764+0000","last_became_active":"2024-09-16T14:07:22.346031+0000","last_became_peered":"2024-09-16T14:07:22.346031+0000","last_unstale":"2024-09-16T14:07:29.472764+0000","last_undegraded":"2024-09-16T14:07:29.472764+0000","last_fullsized":"2024-09-16T14:07:29.472764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:39.265638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003350142,"stat_sum":{"num_bytes":1688787,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":189,"num_read_kb":617,"num_write":39,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"41'105","reported_seq":559,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.422277+0000","last_change":"2024-09-16T14:07:11.338172+0000","last_active":"2024-09-16T14:07:29.422277+0000","last_peered":"2024-09-16T14:07:29.422277+0000","last_clean":"2024-09-16T14:07:29.422277+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:07:29.422277+0000","last_undegraded":"2024-09-16T14:07:29.422277+0000","last_fullsized":"2024-09-16T14:07:29.422277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148865,"stat_sum":{"num_bytes":7460404,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":911,"num_read_kb":2740,"num_write":118,"num_write_kb":1705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"41'591","reported_seq":690,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.476911+0000","last_change":"2024-09-16T14:07:11.338594+0000","last_active":"2024-09-16T14:07:29.476911+0000","last_peered":"2024-09-16T14:07:29.476911+0000","last_clean":"2024-09-16T14:07:29.476911+0000","last_became_active":"2024-09-16T14:07:09.551606+0000","last_became_peered":"2024-09-16T14:07:09.551606+0000","last_unstale":"2024-09-16T14:07:29.476911+0000","last_undegraded":"2024-09-16T14:07:29.476911+0000","last_fullsized":"2024-09-16T14:07:29.476911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:52.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215788,"stat_sum":{"num_bytes":1887490,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":3148,"num_read_kb":2718,"num_write":1062,"num_write_kb":1207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"41'57","reported_seq":299,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.459621+0000","last_change":"2024-09-16T14:07:11.361262+0000","last_active":"2024-09-16T14:07:29.459621+0000","last_peered":"2024-09-16T14:07:29.459621+0000","last_clean":"2024-09-16T14:07:29.459621+0000","last_became_active":"2024-09-16T14:07:09.554272+0000","last_became_peered":"2024-09-16T14:07:09.554272+0000","last_unstale":"2024-09-16T14:07:29.459621+0000","last_undegraded":"2024-09-16T14:07:29.459621+0000","last_fullsized":"2024-09-16T14:07:29.459621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:10.794154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500454,"stat_sum":{"num_bytes":10989994,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":235,"num_read_kb":1034,"num_write":43,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"41'78","reported_seq":365,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.470601+0000","last_change":"2024-09-16T14:07:11.343447+0000","last_active":"2024-09-16T14:07:29.470601+0000","last_peered":"2024-09-16T14:07:29.470601+0000","last_clean":"2024-09-16T14:07:29.470601+0000","last_became_active":"2024-09-16T14:07:09.549545+0000","last_became_peered":"2024-09-16T14:07:09.549545+0000","last_unstale":"2024-09-16T14:07:29.470601+0000","last_undegraded":"2024-09-16T14:07:29.470601+0000","last_fullsized":"2024-09-16T14:07:29.470601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:27.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012153120000000001,"stat_sum":{"num_bytes":20221001,"num_objects":25,"num_object_clones":9,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":298,"num_read_kb":2405,"num_write":128,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"42'137","reported_seq":821,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.724285+0000","last_change":"2024-09-16T14:07:11.546981+0000","last_active":"2024-09-16T14:07:31.724285+0000","last_peered":"2024-09-16T14:07:31.724285+0000","last_clean":"2024-09-16T14:07:31.724285+0000","last_became_active":"2024-09-16T14:07:09.552582+0000","last_became_peered":"2024-09-16T14:07:09.552582+0000","last_unstale":"2024-09-16T14:07:31.724285+0000","last_undegraded":"2024-09-16T14:07:31.724285+0000","last_fullsized":"2024-09-16T14:07:31.724285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:08.157994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009984429999999999,"stat_sum":{"num_bytes":16881183,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":1492,"num_read_kb":1816,"num_write":123,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.676198+0000","last_change":"2024-09-16T14:07:02.504261+0000","last_active":"2024-09-16T14:07:28.676198+0000","last_peered":"2024-09-16T14:07:28.676198+0000","last_clean":"2024-09-16T14:07:28.676198+0000","last_became_active":"2024-09-16T14:07:00.475877+0000","last_became_peered":"2024-09-16T14:07:00.475877+0000","last_unstale":"2024-09-16T14:07:28.676198+0000","last_undegraded":"2024-09-16T14:07:28.676198+0000","last_fullsized":"2024-09-16T14:07:28.676198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:15.110271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020129200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"45'189","reported_seq":407,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T14:07:35.673837+0000","last_change":"2024-09-16T14:07:11.370476+0000","last_active":"2024-09-16T14:07:35.673837+0000","last_peered":"2024-09-16T14:07:35.673837+0000","last_clean":"2024-09-16T14:07:35.673837+0000","last_became_active":"2024-09-16T14:07:09.544524+0000","last_became_peered":"2024-09-16T14:07:09.544524+0000","last_unstale":"2024-09-16T14:07:35.673837+0000","last_undegraded":"2024-09-16T14:07:35.673837+0000","last_fullsized":"2024-09-16T14:07:35.673837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:59.087380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032490359999999999,"stat_sum":{"num_bytes":3500616,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":866,"num_read_kb":690,"num_write":267,"num_write_kb":513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.671019+0000","last_change":"2024-09-16T14:07:22.350069+0000","last_active":"2024-09-16T14:07:28.671019+0000","last_peered":"2024-09-16T14:07:28.671019+0000","last_clean":"2024-09-16T14:07:28.671019+0000","last_became_active":"2024-09-16T14:07:22.349757+0000","last_became_peered":"2024-09-16T14:07:22.349757+0000","last_unstale":"2024-09-16T14:07:28.671019+0000","last_undegraded":"2024-09-16T14:07:28.671019+0000","last_fullsized":"2024-09-16T14:07:28.671019+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:57.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"42'72","reported_seq":321,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.607180+0000","last_change":"2024-09-16T14:07:22.351951+0000","last_active":"2024-09-16T14:07:31.607180+0000","last_peered":"2024-09-16T14:07:31.607180+0000","last_clean":"2024-09-16T14:07:31.607180+0000","last_became_active":"2024-09-16T14:07:22.351394+0000","last_became_peered":"2024-09-16T14:07:22.351394+0000","last_unstale":"2024-09-16T14:07:31.607180+0000","last_undegraded":"2024-09-16T14:07:31.607180+0000","last_fullsized":"2024-09-16T14:07:31.607180+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:55.945287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027484229999999998,"stat_sum":{"num_bytes":21814513,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":256,"num_read_kb":7258,"num_write":85,"num_write_kb":11743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.676097+0000","last_change":"2024-09-16T14:07:02.502729+0000","last_active":"2024-09-16T14:07:28.676097+0000","last_peered":"2024-09-16T14:07:28.676097+0000","last_clean":"2024-09-16T14:07:28.676097+0000","last_became_active":"2024-09-16T14:07:00.476599+0000","last_became_peered":"2024-09-16T14:07:00.476599+0000","last_unstale":"2024-09-16T14:07:28.676097+0000","last_undegraded":"2024-09-16T14:07:28.676097+0000","last_fullsized":"2024-09-16T14:07:28.676097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:38.332613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056022899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"41'67","reported_seq":322,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.427894+0000","last_change":"2024-09-16T14:07:11.547417+0000","last_active":"2024-09-16T14:07:29.427894+0000","last_peered":"2024-09-16T14:07:29.427894+0000","last_clean":"2024-09-16T14:07:29.427894+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:07:29.427894+0000","last_undegraded":"2024-09-16T14:07:29.427894+0000","last_fullsized":"2024-09-16T14:07:29.427894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097608199999999997,"stat_sum":{"num_bytes":1804122,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":250,"num_read_kb":994,"num_write":54,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.662515+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:07:28.662515+0000","last_peered":"2024-09-16T14:07:28.662515+0000","last_clean":"2024-09-16T14:07:28.662515+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:07:28.662515+0000","last_undegraded":"2024-09-16T14:07:28.662515+0000","last_fullsized":"2024-09-16T14:07:28.662515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"41'98","reported_seq":386,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.475248+0000","last_change":"2024-09-16T14:07:11.361331+0000","last_active":"2024-09-16T14:07:29.475248+0000","last_peered":"2024-09-16T14:07:29.475248+0000","last_clean":"2024-09-16T14:07:29.475248+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:07:29.475248+0000","last_undegraded":"2024-09-16T14:07:29.475248+0000","last_fullsized":"2024-09-16T14:07:29.475248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062220600000000004,"stat_sum":{"num_bytes":7015650,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":10,"num_read":420,"num_read_kb":1255,"num_write":107,"num_write_kb":497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"42'290","reported_seq":1280,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:30.972540+0000","last_change":"2024-09-16T14:07:11.342816+0000","last_active":"2024-09-16T14:07:30.972540+0000","last_peered":"2024-09-16T14:07:30.972540+0000","last_clean":"2024-09-16T14:07:30.972540+0000","last_became_active":"2024-09-16T14:07:09.549842+0000","last_became_peered":"2024-09-16T14:07:09.549842+0000","last_unstale":"2024-09-16T14:07:30.972540+0000","last_undegraded":"2024-09-16T14:07:30.972540+0000","last_fullsized":"2024-09-16T14:07:30.972540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:47.724999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014467779999999999,"stat_sum":{"num_bytes":8230979,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":2979,"num_read_kb":2997,"num_write":346,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.676139+0000","last_change":"2024-09-16T14:07:02.503836+0000","last_active":"2024-09-16T14:07:28.676139+0000","last_peered":"2024-09-16T14:07:28.676139+0000","last_clean":"2024-09-16T14:07:28.676139+0000","last_became_active":"2024-09-16T14:07:00.572358+0000","last_became_peered":"2024-09-16T14:07:00.572358+0000","last_unstale":"2024-09-16T14:07:28.676139+0000","last_undegraded":"2024-09-16T14:07:28.676139+0000","last_fullsized":"2024-09-16T14:07:28.676139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:28.110370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673887,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"42'89","reported_seq":359,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.266740+0000","last_change":"2024-09-16T14:07:11.337642+0000","last_active":"2024-09-16T14:07:31.266740+0000","last_peered":"2024-09-16T14:07:31.266740+0000","last_clean":"2024-09-16T14:07:31.266740+0000","last_became_active":"2024-09-16T14:07:09.543015+0000","last_became_peered":"2024-09-16T14:07:09.543015+0000","last_unstale":"2024-09-16T14:07:31.266740+0000","last_undegraded":"2024-09-16T14:07:31.266740+0000","last_fullsized":"2024-09-16T14:07:31.266740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:22:22.438444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076943300000000003,"stat_sum":{"num_bytes":6931999,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":277,"num_read_kb":2115,"num_write":100,"num_write_kb":1793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.797906+0000","last_change":"2024-09-16T14:07:04.121594+0000","last_active":"2024-09-16T14:07:28.797906+0000","last_peered":"2024-09-16T14:07:28.797906+0000","last_clean":"2024-09-16T14:07:28.797906+0000","last_became_active":"2024-09-16T14:07:00.475232+0000","last_became_peered":"2024-09-16T14:07:00.475232+0000","last_unstale":"2024-09-16T14:07:28.797906+0000","last_undegraded":"2024-09-16T14:07:28.797906+0000","last_fullsized":"2024-09-16T14:07:28.797906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.457456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068024199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"41'86","reported_seq":522,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:30.084347+0000","last_change":"2024-09-16T14:07:11.547270+0000","last_active":"2024-09-16T14:07:30.084347+0000","last_peered":"2024-09-16T14:07:30.084347+0000","last_clean":"2024-09-16T14:07:30.084347+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:07:30.084347+0000","last_undegraded":"2024-09-16T14:07:30.084347+0000","last_fullsized":"2024-09-16T14:07:30.084347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085373199999999995,"stat_sum":{"num_bytes":10717480,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":959,"num_read_kb":1353,"num_write":75,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.675775+0000","last_change":"2024-09-16T14:07:02.502802+0000","last_active":"2024-09-16T14:07:28.675775+0000","last_peered":"2024-09-16T14:07:28.675775+0000","last_clean":"2024-09-16T14:07:28.675775+0000","last_became_active":"2024-09-16T14:07:00.480643+0000","last_became_peered":"2024-09-16T14:07:00.480643+0000","last_unstale":"2024-09-16T14:07:28.675775+0000","last_undegraded":"2024-09-16T14:07:28.675775+0000","last_fullsized":"2024-09-16T14:07:28.675775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:52.536849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"42'75","reported_seq":224,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.267541+0000","last_change":"2024-09-16T14:07:22.350954+0000","last_active":"2024-09-16T14:07:31.267541+0000","last_peered":"2024-09-16T14:07:31.267541+0000","last_clean":"2024-09-16T14:07:31.267541+0000","last_became_active":"2024-09-16T14:07:22.350024+0000","last_became_peered":"2024-09-16T14:07:22.350024+0000","last_unstale":"2024-09-16T14:07:31.267541+0000","last_undegraded":"2024-09-16T14:07:31.267541+0000","last_fullsized":"2024-09-16T14:07:31.267541+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:07.634077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002704522,"stat_sum":{"num_bytes":2439817,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":132,"num_read_kb":294,"num_write":65,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":36,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.675912+0000","last_change":"2024-09-16T14:07:02.503755+0000","last_active":"2024-09-16T14:07:28.675912+0000","last_peered":"2024-09-16T14:07:28.675912+0000","last_clean":"2024-09-16T14:07:28.675912+0000","last_became_active":"2024-09-16T14:07:00.569748+0000","last_became_peered":"2024-09-16T14:07:00.569748+0000","last_unstale":"2024-09-16T14:07:28.675912+0000","last_undegraded":"2024-09-16T14:07:28.675912+0000","last_fullsized":"2024-09-16T14:07:28.675912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:11.927535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015626559999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":123,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:28.665175+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:07:28.665175+0000","last_peered":"2024-09-16T14:07:28.665175+0000","last_clean":"2024-09-16T14:07:28.665175+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:07:28.665175+0000","last_undegraded":"2024-09-16T14:07:28.665175+0000","last_fullsized":"2024-09-16T14:07:28.665175+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"41'88","reported_seq":430,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.694313+0000","last_change":"2024-09-16T14:07:22.352613+0000","last_active":"2024-09-16T14:07:29.694313+0000","last_peered":"2024-09-16T14:07:29.694313+0000","last_clean":"2024-09-16T14:07:29.694313+0000","last_became_active":"2024-09-16T14:07:22.351412+0000","last_became_peered":"2024-09-16T14:07:22.351412+0000","last_unstale":"2024-09-16T14:07:29.694313+0000","last_undegraded":"2024-09-16T14:07:29.694313+0000","last_fullsized":"2024-09-16T14:07:29.694313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:21.471879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022177080000000001,"stat_sum":{"num_bytes":7738782,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":425,"num_read_kb":1090,"num_write":84,"num_write_kb":368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"42'87","reported_seq":412,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.386610+0000","last_change":"2024-09-16T14:07:11.545923+0000","last_active":"2024-09-16T14:07:31.386610+0000","last_peered":"2024-09-16T14:07:31.386610+0000","last_clean":"2024-09-16T14:07:31.386610+0000","last_became_active":"2024-09-16T14:07:09.552836+0000","last_became_peered":"2024-09-16T14:07:09.552836+0000","last_unstale":"2024-09-16T14:07:31.386610+0000","last_undegraded":"2024-09-16T14:07:31.386610+0000","last_fullsized":"2024-09-16T14:07:31.386610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:41.628255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339700000000001,"stat_sum":{"num_bytes":6942792,"num_objects":23,"num_object_clones":9,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":336,"num_read_kb":2230,"num_write":117,"num_write_kb":2532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"42'191","reported_seq":464,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.686609+0000","last_change":"2024-09-16T14:07:11.545988+0000","last_active":"2024-09-16T14:07:31.686609+0000","last_peered":"2024-09-16T14:07:31.686609+0000","last_clean":"2024-09-16T14:07:31.686609+0000","last_became_active":"2024-09-16T14:07:09.550304+0000","last_became_peered":"2024-09-16T14:07:09.550304+0000","last_unstale":"2024-09-16T14:07:31.686609+0000","last_undegraded":"2024-09-16T14:07:31.686609+0000","last_fullsized":"2024-09-16T14:07:31.686609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:42.431894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065751299999999998,"stat_sum":{"num_bytes":4534327,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":915,"num_read_kb":953,"num_write":228,"num_write_kb":409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"41'45","reported_seq":210,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.364518+0000","last_change":"2024-09-16T14:07:11.377884+0000","last_active":"2024-09-16T14:07:29.364518+0000","last_peered":"2024-09-16T14:07:29.364518+0000","last_clean":"2024-09-16T14:07:29.364518+0000","last_became_active":"2024-09-16T14:07:09.544775+0000","last_became_peered":"2024-09-16T14:07:09.544775+0000","last_unstale":"2024-09-16T14:07:29.364518+0000","last_undegraded":"2024-09-16T14:07:29.364518+0000","last_fullsized":"2024-09-16T14:07:29.364518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:16.336123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056925200000000002,"stat_sum":{"num_bytes":6116432,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":154,"num_read_kb":449,"num_write":35,"num_write_kb":295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"42'56","reported_seq":251,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.080039+0000","last_change":"2024-09-16T14:07:11.343448+0000","last_active":"2024-09-16T14:07:31.080039+0000","last_peered":"2024-09-16T14:07:31.080039+0000","last_clean":"2024-09-16T14:07:31.080039+0000","last_became_active":"2024-09-16T14:07:09.553360+0000","last_became_peered":"2024-09-16T14:07:09.553360+0000","last_unstale":"2024-09-16T14:07:31.080039+0000","last_undegraded":"2024-09-16T14:07:31.080039+0000","last_fullsized":"2024-09-16T14:07:31.080039+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:28.854037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017881959999999999,"stat_sum":{"num_bytes":2699704,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":194,"num_read_kb":912,"num_write":41,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"42'41","reported_seq":112,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.128753+0000","last_change":"2024-09-16T14:07:11.546511+0000","last_active":"2024-09-16T14:07:31.128753+0000","last_peered":"2024-09-16T14:07:31.128753+0000","last_clean":"2024-09-16T14:07:31.128753+0000","last_became_active":"2024-09-16T14:07:09.559703+0000","last_became_peered":"2024-09-16T14:07:09.559703+0000","last_unstale":"2024-09-16T14:07:31.128753+0000","last_undegraded":"2024-09-16T14:07:31.128753+0000","last_fullsized":"2024-09-16T14:07:31.128753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:06.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074836900000000003,"stat_sum":{"num_bytes":2026071,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":49,"num_read_kb":191,"num_write":35,"num_write_kb":405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"41'75","reported_seq":554,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.921183+0000","last_change":"2024-09-16T14:07:11.362189+0000","last_active":"2024-09-16T14:07:29.921183+0000","last_peered":"2024-09-16T14:07:29.921183+0000","last_clean":"2024-09-16T14:07:29.921183+0000","last_became_active":"2024-09-16T14:07:09.548973+0000","last_became_peered":"2024-09-16T14:07:09.548973+0000","last_unstale":"2024-09-16T14:07:29.921183+0000","last_undegraded":"2024-09-16T14:07:29.921183+0000","last_fullsized":"2024-09-16T14:07:29.921183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:17.345008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013505660000000001,"stat_sum":{"num_bytes":9781675,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":1115,"num_read_kb":948,"num_write":67,"num_write_kb":716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"42'276","reported_seq":637,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.706063+0000","last_change":"2024-09-16T14:07:11.342735+0000","last_active":"2024-09-16T14:07:31.706063+0000","last_peered":"2024-09-16T14:07:31.706063+0000","last_clean":"2024-09-16T14:07:31.706063+0000","last_became_active":"2024-09-16T14:07:09.559054+0000","last_became_peered":"2024-09-16T14:07:09.559054+0000","last_unstale":"2024-09-16T14:07:31.706063+0000","last_undegraded":"2024-09-16T14:07:31.706063+0000","last_fullsized":"2024-09-16T14:07:31.706063+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.926445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372222,"stat_sum":{"num_bytes":14092547,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":3,"num_read":1868,"num_read_kb":2785,"num_write":419,"num_write_kb":5199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"41'57","reported_seq":252,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.794768+0000","last_change":"2024-09-16T14:07:11.340733+0000","last_active":"2024-09-16T14:07:29.794768+0000","last_peered":"2024-09-16T14:07:29.794768+0000","last_clean":"2024-09-16T14:07:29.794768+0000","last_became_active":"2024-09-16T14:07:09.540341+0000","last_became_peered":"2024-09-16T14:07:09.540341+0000","last_unstale":"2024-09-16T14:07:29.794768+0000","last_undegraded":"2024-09-16T14:07:29.794768+0000","last_fullsized":"2024-09-16T14:07:29.794768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:18.953079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012161470000000001,"stat_sum":{"num_bytes":6660096,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":188,"num_read_kb":2027,"num_write":70,"num_write_kb":1227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"42'62","reported_seq":184,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.608079+0000","last_change":"2024-09-16T14:07:11.545567+0000","last_active":"2024-09-16T14:07:31.608079+0000","last_peered":"2024-09-16T14:07:31.608079+0000","last_clean":"2024-09-16T14:07:31.608079+0000","last_became_active":"2024-09-16T14:07:09.556146+0000","last_became_peered":"2024-09-16T14:07:09.556146+0000","last_unstale":"2024-09-16T14:07:31.608079+0000","last_undegraded":"2024-09-16T14:07:31.608079+0000","last_fullsized":"2024-09-16T14:07:31.608079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:29.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037127800000000001,"stat_sum":{"num_bytes":7409265,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":123,"num_read_kb":319,"num_write":61,"num_write_kb":5102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"42'73","reported_seq":110,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T14:07:31.555960+0000","last_change":"2024-09-16T14:07:11.547712+0000","last_active":"2024-09-16T14:07:31.555960+0000","last_peered":"2024-09-16T14:07:31.555960+0000","last_clean":"2024-09-16T14:07:31.555960+0000","last_became_active":"2024-09-16T14:07:09.553092+0000","last_became_peered":"2024-09-16T14:07:09.553092+0000","last_unstale":"2024-09-16T14:07:31.555960+0000","last_undegraded":"2024-09-16T14:07:31.555960+0000","last_fullsized":"2024-09-16T14:07:31.555960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:54.650445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012662420000000001,"stat_sum":{"num_bytes":9044575,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":20,"num_read_kb":34,"num_write":67,"num_write_kb":521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"41'78","reported_seq":366,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T14:07:29.439918+0000","last_change":"2024-09-16T14:07:11.546794+0000","last_active":"2024-09-16T14:07:29.439918+0000","last_peered":"2024-09-16T14:07:29.439918+0000","last_clean":"2024-09-16T14:07:29.439918+0000","last_became_active":"2024-09-16T14:07:09.555363+0000","last_became_peered":"2024-09-16T14:07:09.555363+0000","last_unstale":"2024-09-16T14:07:29.439918+0000","last_undegraded":"2024-09-16T14:07:29.439918+0000","last_fullsized":"2024-09-16T14:07:29.439918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:05.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066482800000000001,"stat_sum":{"num_bytes":14949208,"num_objects":16,"num_object_clones":7,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":284,"num_read_kb":3234,"num_write":126,"num_write_kb":2551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":270825051,"num_objects":596,"num_object_clones":169,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":187,"num_read":22898,"num_read_kb":55868,"num_write":4930,"num_write_kb":64190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10067968,"data_stored":69680684,"data_compressed":2874698,"data_compressed_allocated":6864896,"data_compressed_original":67507616,"omap_allocated":0,"internal_metadata":0},"log_size":3856,"ondisk_log_size":3856,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302636,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069204,"statfs":{"total":96636764160,"available":96326864896,"internally_reserved":0,"allocated":1134592,"data_stored":3042220,"data_compressed":168116,"data_compressed_allocated":348160,"data_compressed_original":2633728,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303508,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068332,"statfs":{"total":96636764160,"available":96325971968,"internally_reserved":0,"allocated":2039808,"data_stored":12827575,"data_compressed":435397,"data_compressed_allocated":1187840,"data_compressed_original":12401614,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]}]},{"osd":5,"up_from":15,"seq":64424509453,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307884,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063956,"statfs":{"total":96636764160,"available":96321490944,"internally_reserved":0,"allocated":2260992,"data_stored":14817339,"data_compressed":511764,"data_compressed_allocated":1343488,"data_compressed_original":14303232,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307460,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064380,"statfs":{"total":96636764160,"available":96321925120,"internally_reserved":0,"allocated":1822720,"data_stored":7541599,"data_compressed":520120,"data_compressed_allocated":884736,"data_compressed_original":7044866,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302800,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069040,"statfs":{"total":96636764160,"available":96326696960,"internally_reserved":0,"allocated":1314816,"data_stored":1277982,"data_compressed":315189,"data_compressed_allocated":385024,"data_compressed_original":786432,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303512,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068328,"statfs":{"total":96636764160,"available":96325967872,"internally_reserved":0,"allocated":2048000,"data_stored":12882981,"data_compressed":439090,"data_compressed_allocated":1187840,"data_compressed_original":12431360,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]}]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303528,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068312,"statfs":{"total":96636764160,"available":96325951488,"internally_reserved":0,"allocated":2064384,"data_stored":9992993,"data_compressed":448986,"data_compressed_allocated":1015808,"data_compressed_original":9395970,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]}]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303180,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94068660,"statfs":{"total":96636764160,"available":96326307840,"internally_reserved":0,"allocated":1699840,"data_stored":10340663,"data_compressed":287106,"data_compressed_allocated":856064,"data_compressed_original":9915342,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1564672,"data_stored":9666904,"data_compressed":451386,"data_compressed_allocated":1007616,"data_compressed_original":9277186,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1433600,"data_stored":11923163,"data_compressed":385076,"data_compressed_allocated":1093632,"data_compressed_original":11694080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":761856,"data_stored":1013224,"data_compressed":269178,"data_compressed_allocated":335872,"data_compressed_original":737280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1273856,"data_stored":7205659,"data_compressed":491945,"data_compressed_allocated":847872,"data_compressed_original":6926082,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1380352,"data_stored":11028211,"data_compressed":293094,"data_compressed_allocated":921600,"data_compressed_original":10718158,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1552384,"data_stored":13676928,"data_compressed":419224,"data_compressed_allocated":1191936,"data_compressed_original":13402112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1568768,"data_stored":12633511,"data_compressed":435397,"data_compressed_allocated":1187840,"data_compressed_original":12401614,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":2533084,"data_compressed":129398,"data_compressed_allocated":278528,"data_compressed_original":2351104,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:07:37.435 INFO:tasks.thrashosds.thrasher:Setting 3.1d to [5, 3, 4, 1] 2024-09-16T14:07:37.435 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1d', '5', '3', '4', '1'] 2024-09-16T14:07:37.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1d 5 3 4 1 2024-09-16T14:07:37.456 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:37.456 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:37.465 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:37.474 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:37.474 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:37.470+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:37.497 INFO:teuthology.orchestra.run.smithi079.stdout:1499 writesame 0xce8f1e2 thru 0xce914f8 data_size 0x1(0x2317 bytes) 2024-09-16T14:07:37.497 INFO:teuthology.orchestra.run.smithi079.stdout:1500 write 0x315c33 thru 0x3246cb (0xea99 bytes) 2024-09-16T14:07:37.524 INFO:teuthology.orchestra.run.smithi079.stdout:1501 read 0x670c7ab thru 0x6719d63 (0xd5b9 bytes) 2024-09-16T14:07:37.528 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:37.528 INFO:teuthology.orchestra.run.smithi079.stdout:1503 punch from 0x9588b17 to 0x9594341, (0xb82a bytes) 2024-09-16T14:07:37.530 INFO:teuthology.orchestra.run.smithi079.stdout:1504 writesame 0x503413 thru 0x507cfd data_size 0x1(0x48eb bytes) 2024-09-16T14:07:37.538 INFO:teuthology.orchestra.run.smithi079.stdout:1506 read 0x59561b3 thru 0x595a1e7 (0x4035 bytes) 2024-09-16T14:07:37.609 INFO:teuthology.orchestra.run.smithi079.stdout:1507 read 0x83ae67 thru 0x8436b1 (0x884b bytes) 2024-09-16T14:07:37.610 INFO:teuthology.orchestra.run.smithi079.stdout:1508 punch from 0x854e5dc to 0x8557841, (0x9265 bytes) 2024-09-16T14:07:37.610 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:37.610 INFO:teuthology.orchestra.run.smithi079.stdout:1510 read 0xae557e8 thru 0xae5888a (0x30a3 bytes) 2024-09-16T14:07:37.610 INFO:teuthology.orchestra.run.smithi079.stdout:1511 read 0xbd017ff thru 0xbd0c2f2 (0xaaf4 bytes) 2024-09-16T14:07:37.610 INFO:teuthology.orchestra.run.smithi079.stdout:1512 trunc from 0xce914f9 to 0x70f44fa 2024-09-16T14:07:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:37.610 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:37.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:37.611 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:37.602+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:37.696 INFO:teuthology.orchestra.run.smithi079.stdout:1513 punch from 0x45e3815 to 0x45ede12, (0xa5fd bytes) 2024-09-16T14:07:37.696 INFO:teuthology.orchestra.run.smithi079.stdout:1514 punch from 0x2305dbb to 0x2306a8f, (0xcd4 bytes) 2024-09-16T14:07:37.697 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:37.697 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:37.707 INFO:teuthology.orchestra.run.smithi079.stdout:1517 trunc from 0x70f44fa to 0x565153 2024-09-16T14:07:37.709 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:37.702+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:37.747 INFO:teuthology.orchestra.run.smithi079.stdout:1518 read 0x197a0 thru 0x23be3 (0xa444 bytes) 2024-09-16T14:07:37.777 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:37.777 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:37.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:37.811 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:37.806+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:37.912 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:37.906+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:37.921 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:37.926 INFO:teuthology.orchestra.run.smithi079.stdout:1519 write 0xaf3507d thru 0xaf36283 (0x1207 bytes) 2024-09-16T14:07:37.937 INFO:teuthology.orchestra.run.smithi079.stdout:1520 write 0xa400266 thru 0xa40e8a5 (0xe640 bytes) 2024-09-16T14:07:37.955 INFO:teuthology.orchestra.run.smithi079.stdout:1521 write 0x1abb9a4 thru 0x1ac9365 (0xd9c2 bytes) 2024-09-16T14:07:37.973 INFO:teuthology.orchestra.run.smithi079.stdout:1522 write 0x37f16c9 thru 0x37f717b (0x5ab3 bytes) 2024-09-16T14:07:37.999 INFO:teuthology.orchestra.run.smithi079.stdout:1524 punch from 0x6360646 to 0x6364bdd, (0x4597 bytes) 2024-09-16T14:07:37.999 INFO:teuthology.orchestra.run.smithi079.stdout:1526 write 0x267b888 thru 0x267f9b8 (0x4131 bytes) 2024-09-16T14:07:38.012 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:38.006+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:38.037 INFO:teuthology.orchestra.run.smithi079.stdout:1527 write 0xd885bb2 thru 0xd891193 (0xb5e2 bytes) 2024-09-16T14:07:38.060 INFO:teuthology.orchestra.run.smithi079.stdout:1528 write 0x85b206 thru 0x86582e (0xa629 bytes) 2024-09-16T14:07:38.068 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:38.068 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:38.080 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:38.081 INFO:teuthology.orchestra.run.smithi079.stdout:1529 punch from 0xc82f7a1 to 0xc839e10, (0xa66f bytes) 2024-09-16T14:07:38.081 INFO:teuthology.orchestra.run.smithi079.stdout:1530 writesame 0x577f76e thru 0x57869cd data_size 0x30(0x7260 bytes) 2024-09-16T14:07:38.099 INFO:teuthology.orchestra.run.smithi079.stdout:1532 write 0x7829f68 thru 0x782f3af (0x5448 bytes) 2024-09-16T14:07:38.113 INFO:teuthology.orchestra.run.smithi079.stdout:1533 read 0x6421c79 thru 0x642d5e6 (0xb96e bytes) 2024-09-16T14:07:38.114 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:38.106+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:38.132 INFO:teuthology.orchestra.run.smithi079.stdout:1535 trunc from 0xd891194 to 0x3d77370 2024-09-16T14:07:38.173 INFO:teuthology.orchestra.run.smithi079.stdout:1536 trunc from 0x3d77370 to 0xc06e003 2024-09-16T14:07:38.179 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:38.179 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:38.181 INFO:teuthology.orchestra.run.smithi079.stdout:1539 writesame 0x5bb6321 thru 0x5bbffc2 data_size 0x7b(0x9ca2 bytes) 2024-09-16T14:07:38.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:38.201 INFO:teuthology.orchestra.run.smithi079.stdout:1540 punch from 0x2d929e7 to 0x2d9dc1d, (0xb236 bytes) 2024-09-16T14:07:38.208 INFO:teuthology.orchestra.run.smithi079.stdout:1542 write 0xc270144 thru 0xc2793b9 (0x9276 bytes) 2024-09-16T14:07:38.214 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:38.210+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:38.230 INFO:teuthology.orchestra.run.smithi079.stdout:1544 read 0x8e861f7 thru 0x8e8aca7 (0x4ab1 bytes) 2024-09-16T14:07:38.242 INFO:teuthology.orchestra.run.smithi079.stdout:1545 trunc from 0xc2793ba to 0x54eaecc 2024-09-16T14:07:38.265 INFO:teuthology.orchestra.run.smithi079.stdout:1546 writesame 0xab3871 thru 0xab8336 data_size 0x11(0x4ac6 bytes) 2024-09-16T14:07:38.299 INFO:teuthology.orchestra.run.smithi079.stdout:1547 write 0xdfa4b07 thru 0xdfae19a (0x9694 bytes) 2024-09-16T14:07:38.312 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:38.313 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:38.315 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:38.310+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:38.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:38.327 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:38.327 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:38.327 INFO:teuthology.orchestra.run.smithi079.stdout:1550 read 0x4352cc1 thru 0x4354722 (0x1a62 bytes) 2024-09-16T14:07:38.328 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:38.328 INFO:teuthology.orchestra.run.smithi079.stdout:1552 read 0x42e395 thru 0x4346b5 (0x6321 bytes) 2024-09-16T14:07:38.331 INFO:teuthology.orchestra.run.smithi079.stdout:1554 trunc from 0xdfae19b to 0xc41166e 2024-09-16T14:07:38.353 INFO:teuthology.orchestra.run.smithi079.stdout:1555 trunc from 0xc41166e to 0xdaf9aa1 2024-09-16T14:07:38.362 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:38.372 INFO:teuthology.orchestra.run.smithi079.stdout:1557 trunc from 0xdaf9aa1 to 0x7768d3e 2024-09-16T14:07:38.383 INFO:teuthology.orchestra.run.smithi079.stdout:1558 write 0x6c1127c thru 0x6c1ca73 (0xb7f8 bytes) 2024-09-16T14:07:38.415 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:38.410+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:38.417 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:38.427 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:38.427 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:38.433 INFO:teuthology.orchestra.run.smithi079.stdout:1560 writesame 0xba2f32d thru 0xba3766b data_size 0x1(0x833f bytes) 2024-09-16T14:07:38.434 INFO:teuthology.orchestra.run.smithi079.stdout:1561 write 0x8405f2b thru 0x840da47 (0x7b1d bytes) 2024-09-16T14:07:38.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:38.460 INFO:teuthology.orchestra.run.smithi079.stdout:1562 writesame 0x46ed2a7 thru 0x46ef5b8 data_size 0x2(0x2312 bytes) 2024-09-16T14:07:38.483 INFO:teuthology.orchestra.run.smithi079.stdout:1564 read 0x7c4cf87 thru 0x7c5b29e (0xe318 bytes) 2024-09-16T14:07:38.483 INFO:teuthology.orchestra.run.smithi079.stdout:1565 write 0x5963180 thru 0x596b3f7 (0x8278 bytes) 2024-09-16T14:07:38.503 INFO:teuthology.orchestra.run.smithi079.stdout:1566 read 0x85944c7 thru 0x85996bf (0x51f9 bytes) 2024-09-16T14:07:38.504 INFO:teuthology.orchestra.run.smithi079.stdout:1568 write 0x207bc2f thru 0x207c37b (0x74d bytes) 2024-09-16T14:07:38.514 INFO:teuthology.orchestra.run.smithi079.stdout:1569 trunc from 0xba3766c to 0x8e826d0 2024-09-16T14:07:38.516 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:38.510+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:38.524 INFO:teuthology.orchestra.run.smithi079.stdout:1570 punch from 0x29bfc0b to 0x29ca9b1, (0xada6 bytes) 2024-09-16T14:07:38.524 INFO:teuthology.orchestra.run.smithi079.stdout:1571 write 0x24b0669 thru 0x24bfb90 (0xf528 bytes) 2024-09-16T14:07:38.530 INFO:teuthology.orchestra.run.smithi079.stdout:1573 read 0x5ce394b thru 0x5cee2a5 (0xa95b bytes) 2024-09-16T14:07:38.539 INFO:teuthology.orchestra.run.smithi079.stdout:1574 trunc from 0x8e826d0 to 0x3913b10 2024-09-16T14:07:38.574 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:38.574 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:38.581 INFO:teuthology.orchestra.run.smithi079.stdout:1576 write 0x5131a23 thru 0x513a8f4 (0x8ed2 bytes) 2024-09-16T14:07:38.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:38.617 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:38.610+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:38.622 INFO:teuthology.orchestra.run.smithi079.stdout:1579 write 0x7546691 thru 0x754bdb4 (0x5724 bytes) 2024-09-16T14:07:38.668 INFO:teuthology.orchestra.run.smithi079.stdout:1580 write 0x93249f4 thru 0x932ead4 (0xa0e1 bytes) 2024-09-16T14:07:38.697 INFO:teuthology.orchestra.run.smithi079.stdout:1583 read 0x872ba6a thru 0x873aa4d (0xefe4 bytes) 2024-09-16T14:07:38.697 INFO:teuthology.orchestra.run.smithi079.stdout:1584 read 0x6675263 thru 0x6681d35 (0xcad3 bytes) 2024-09-16T14:07:38.698 INFO:teuthology.orchestra.run.smithi079.stdout:1585 read 0x4e9feb8 thru 0x4ea8839 (0x8982 bytes) 2024-09-16T14:07:38.698 INFO:teuthology.orchestra.run.smithi079.stdout:1587 read 0x1334138 thru 0x133a952 (0x681b bytes) 2024-09-16T14:07:38.698 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:38.698 INFO:teuthology.orchestra.run.smithi079.stdout:1590 write 0x2cc8bcd thru 0x2ccd897 (0x4ccb bytes) 2024-09-16T14:07:38.717 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:38.710+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:38.720 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:38.731 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:38.732 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:38.739 INFO:teuthology.orchestra.run.smithi079.stdout:1592 write 0xe0eb887 thru 0xe0f971b (0xde95 bytes) 2024-09-16T14:07:38.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:38.748 INFO:teuthology.orchestra.run.smithi079.stderr:set 3.1d pg_upmap_items mapping to [5->3,4->1] 2024-09-16T14:07:38.786 INFO:teuthology.orchestra.run.smithi079.stdout:1593 trunc from 0xe0f971c to 0x12111be 2024-09-16T14:07:38.818 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:38.814+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:38.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:07:38.829 INFO:teuthology.orchestra.run.smithi079.stdout:1595 read 0x982590 thru 0x98d014 (0xaa85 bytes) 2024-09-16T14:07:38.830 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:38.836 INFO:teuthology.orchestra.run.smithi079.stdout:1598 trunc from 0x12111be to 0x4be08f7 2024-09-16T14:07:38.843 INFO:teuthology.orchestra.run.smithi079.stdout:1599 writesame 0x3c9f250 thru 0x3ca1c8e data_size 0x23(0x2a3f bytes) 2024-09-16T14:07:38.871 INFO:teuthology.orchestra.run.smithi079.stdout:1600 read 0x2bfcefb thru 0x2c03322 (0x6428 bytes) 2024-09-16T14:07:38.872 INFO:teuthology.orchestra.run.smithi079.stdout:1601 punch from 0x192bc4a to 0x192ca18, (0xdce bytes) 2024-09-16T14:07:38.872 INFO:teuthology.orchestra.run.smithi079.stdout:1602 writesame 0x237ecac thru 0x238aca8 data_size 0x2b(0xbffd bytes) 2024-09-16T14:07:38.877 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:38.877 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:38.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:38.897 INFO:teuthology.orchestra.run.smithi079.stdout:1605 write 0x552819 thru 0x55c2ea (0x9ad2 bytes) 2024-09-16T14:07:38.919 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:38.914+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:38.921 INFO:teuthology.orchestra.run.smithi079.stdout:1606 write 0xc3046ee thru 0xc3145c0 (0xfed3 bytes) 2024-09-16T14:07:38.946 INFO:teuthology.orchestra.run.smithi079.stdout:1608 write 0xad7fca1 thru 0xad8f4f8 (0xf858 bytes) 2024-09-16T14:07:38.966 INFO:teuthology.orchestra.run.smithi079.stdout:1611 write 0x3f41c7 thru 0x3fe0c4 (0x9efe bytes) 2024-09-16T14:07:38.982 INFO:teuthology.orchestra.run.smithi079.stdout:1612 writesame 0xe32bc42 thru 0xe33a96e data_size 0x25(0xed2d bytes) 2024-09-16T14:07:38.983 INFO:teuthology.orchestra.run.smithi079.stdout:1613 punch from 0x1aa7a6d to 0x1aacd51, (0x52e4 bytes) 2024-09-16T14:07:38.983 INFO:teuthology.orchestra.run.smithi079.stdout:1615 writesame 0x5a759ae thru 0x5a8185c data_size 0xd(0xbeaf bytes) 2024-09-16T14:07:39.006 INFO:teuthology.orchestra.run.smithi079.stdout:1616 write 0xadeec2a thru 0xadf24d1 (0x38a8 bytes) 2024-09-16T14:07:39.008 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:39.009 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:39.010 INFO:teuthology.orchestra.run.smithi079.stdout:1618 punch from 0xd558e1f to 0xd55b42f, (0x2610 bytes) 2024-09-16T14:07:39.017 INFO:teuthology.orchestra.run.smithi079.stdout:1620 write 0xe3a4d1e thru 0xe3b196c (0xcc4f bytes) 2024-09-16T14:07:39.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:39.040 INFO:teuthology.orchestra.run.smithi079.stdout:1621 write 0x48249c8 thru 0x4831bcc (0xd205 bytes) 2024-09-16T14:07:39.040 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:39.034+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:39.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:39.061 INFO:teuthology.orchestra.run.smithi079.stdout:1622 punch from 0x3bbc3e9 to 0x3bc4876, (0x848d bytes) 2024-09-16T14:07:39.061 INFO:teuthology.orchestra.run.smithi079.stdout:1623 write 0xd36ab3f thru 0xd373195 (0x8657 bytes) 2024-09-16T14:07:39.083 INFO:teuthology.orchestra.run.smithi079.stdout:1624 read 0x30ae764 thru 0x30b6882 (0x811f bytes) 2024-09-16T14:07:39.083 INFO:teuthology.orchestra.run.smithi079.stdout:1625 write 0x831d53 thru 0x83b662 (0x9910 bytes) 2024-09-16T14:07:39.105 INFO:teuthology.orchestra.run.smithi079.stdout:1631 write 0x2db96a1 thru 0x2dc1f6a (0x88ca bytes) 2024-09-16T14:07:39.120 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:39.114+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:39.136 INFO:teuthology.orchestra.run.smithi079.stdout:1633 writesame 0xec59b42 thru 0xec5fda5 data_size 0x6(0x6264 bytes) 2024-09-16T14:07:39.136 INFO:teuthology.orchestra.run.smithi079.stdout:1635 write 0xa4df862 thru 0xa4e711a (0x78b9 bytes) 2024-09-16T14:07:39.156 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:39.157 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:39.157 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:39.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:39.180 INFO:teuthology.orchestra.run.smithi079.stdout:1637 trunc from 0xec5fda6 to 0xb3d40 2024-09-16T14:07:39.210 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:39.222 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:39.214+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:39.248 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:39.265 INFO:teuthology.orchestra.run.smithi079.stdout:1638 write 0x5f3c63e thru 0x5f473e2 (0xada5 bytes) 2024-09-16T14:07:39.282 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:39.300 INFO:teuthology.orchestra.run.smithi079.stdout:1639 writesame 0x211a3be thru 0x21220cd data_size 0x5c(0x7d10 bytes) 2024-09-16T14:07:39.305 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:39.306 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:39.306 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:39.307 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:39.307 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:39.307 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:39.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:39.323 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:39.318+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:39.324 INFO:teuthology.orchestra.run.smithi079.stdout:1640 write 0xc55600 thru 0xc609f9 (0xb3fa bytes) 2024-09-16T14:07:39.336 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:39.345 INFO:teuthology.orchestra.run.smithi079.stdout:1641 punch from 0x5112a82 to 0x5120d5e, (0xe2dc bytes) 2024-09-16T14:07:39.346 INFO:teuthology.orchestra.run.smithi079.stdout:1643 punch from 0x48aa669 to 0x48b0604, (0x5f9b bytes) 2024-09-16T14:07:39.346 INFO:teuthology.orchestra.run.smithi079.stdout:1644 write 0xf824e6 thru 0xf8ee61 (0xc97c bytes) 2024-09-16T14:07:39.362 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:39.373 INFO:teuthology.orchestra.run.smithi079.stdout:1645 trunc from 0x5f473e3 to 0x965ebb 2024-09-16T14:07:39.383 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:39.398 INFO:teuthology.orchestra.run.smithi079.stdout:1646 read 0x6239f2 thru 0x62bcb7 (0x82c6 bytes) 2024-09-16T14:07:39.409 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:39.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:39.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:39.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:39.412 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:39.412 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:39.412 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:39.412 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:39.412 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:39.423 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:39.418+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:39.426 INFO:teuthology.orchestra.run.smithi079.stdout:1647 writesame 0xe77a54e thru 0xe783489 data_size 0x67(0x8f3c bytes) 2024-09-16T14:07:39.426 INFO:teuthology.orchestra.run.smithi079.stdout:1650 write 0x494f406 thru 0x495eed5 (0xfad0 bytes) 2024-09-16T14:07:39.436 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:39.436 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:39.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:39.436 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:39.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.436 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12 3.e 3:7276ff7f:::rbd_header.113631a28a63:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-16T14:07:39.436 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.074149135999999, 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056564060000000001, 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357783+0000", 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357785+0000", 2024-09-16T14:07:39.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0779999999999998e-06 2024-09-16T14:07:39.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357794+0000", 2024-09-16T14:07:39.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:07:39.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357811+0000", 2024-09-16T14:07:39.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7300999999999999e-05 2024-09-16T14:07:39.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357825+0000", 2024-09-16T14:07:39.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3959e-05 2024-09-16T14:07:39.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357934+0000", 2024-09-16T14:07:39.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010899 2024-09-16T14:07:39.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op must wait for map", 2024-09-16T14:07:39.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.357942+0000", 2024-09-16T14:07:39.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.182e-06 2024-09-16T14:07:39.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.360955+0000", 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003013258 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361149+0000", 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019401600000000001 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:39.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.361448+0000", 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029863300000000003 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.362788+0000", 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013406480000000001 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363338+0000", 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054941500000000004 2024-09-16T14:07:39.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363400+0000", 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2440999999999994e-05 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:11.363440+0000", 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9734999999999998e-05 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:694 3.e 3:76483abd:::rbd_data.1136e0d3367d.000000000000016c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.657517645999999, 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055514199999999996, 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 694 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774415+0000", 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774417+0000", 2024-09-16T14:07:39.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0509999999999999e-06 2024-09-16T14:07:39.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774424+0000", 2024-09-16T14:07:39.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4229999999999999e-06 2024-09-16T14:07:39.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774443+0000", 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8664999999999998e-05 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774457+0000", 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3974e-05 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774531+0000", 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4208999999999994e-05 2024-09-16T14:07:39.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.774612+0000", 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0438000000000005e-05 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.775728+0000", 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001116414 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.777045+0000", 2024-09-16T14:07:39.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013172139999999999 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779908+0000", 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002863071 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779939+0000", 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0840000000000003e-05 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:15.779966+0000", 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7121e-05 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:1106 3.11 e30/23 3:8da88c9c:::rbd_data.11365ca8c0d.0000000000000014:head v 30'5, mlcod=28'1)", 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.173011305999999, 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054785859999999997, 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258921+0000", 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258922+0000", 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9500000000000001e-07 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258929+0000", 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4560000000000003e-06 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258944+0000", 2024-09-16T14:07:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4548e-05 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.258950+0000", 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9959999999999999e-06 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.259023+0000", 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2805000000000005e-05 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.259056+0000", 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3395000000000001e-05 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.264352+0000", 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052958019999999996 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:17.264400+0000", 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7788999999999999e-05 2024-09-16T14:07:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:3850 3.16 3:6fa6863f:::rbd_data.1136494ba971.000000000000021b:head [set-alloc-hint object_size 262144 write_size 262144,write 58251~7285] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.051301254999998, 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052342250000000003, 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3850 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380631+0000", 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380633+0000", 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8920000000000001e-06 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380647+0000", 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3455000000000001e-05 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380670+0000", 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3215999999999999e-05 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380678+0000", 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2779999999999993e-06 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380734+0000", 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6255999999999998e-05 2024-09-16T14:07:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380799+0000", 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4204999999999999e-05 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.380900+0000", 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101672 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385645+0000", 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047443859999999997 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385755+0000", 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000110315 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385815+0000", 2024-09-16T14:07:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0035999999999999e-05 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:20.385865+0000", 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0513999999999998e-05 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:4098 3.1e e35/23 3:783d22fa:::rbd_data.1136494ba971.0000000000000017:head v 35'28, mlcod=35'22)", 2024-09-16T14:07:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.007658845000002, 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00715775, 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424274+0000", 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6499999999999998e-07 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424283+0000", 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6840000000000002e-06 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424295+0000", 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2074e-05 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424302+0000", 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.489e-06 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424345+0000", 2024-09-16T14:07:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2846999999999998e-05 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.424364+0000", 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9034000000000001e-05 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431403+0000", 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070398259999999999 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431431+0000", 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8031e-05 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4102 3.16 3:6ee147bb:::rbd_data.1136494ba971.000000000000001b:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.006894999, 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063615549999999996, 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4102 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425037+0000", 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425038+0000", 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0400000000000005e-07 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425042+0000", 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9090000000000002e-06 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425056+0000", 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3308e-05 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425060+0000", 2024-09-16T14:07:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7820000000000001e-06 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425105+0000", 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4833000000000003e-05 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425174+0000", 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9125e-05 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.425313+0000", 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013855900000000001 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426552+0000", 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00123953 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431313+0000", 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004760401 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431361+0000", 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8535999999999997e-05 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.431399+0000", 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7667999999999999e-05 2024-09-16T14:07:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4105 3.5 3:a4636173:::rbd_data.1136494ba971.000000000000001e:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.005199586, 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0078170070000000008, 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4105 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426733+0000", 2024-09-16T14:07:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426734+0000", 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.088e-06 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426739+0000", 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8550000000000001e-06 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426749+0000", 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0336999999999999e-05 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426755+0000", 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426798+0000", 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2579e-05 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426862+0000", 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3493999999999999e-05 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.426974+0000", 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112271 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.429506+0000", 2024-09-16T14:07:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025319439999999999 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434430+0000", 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049244240000000002 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434501+0000", 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1023000000000001e-05 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.434550+0000", 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8678999999999999e-05 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4127 3.5 3:a4205384:::rbd_data.1136494ba971.0000000000000032:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.995214111999999, 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054291670000000004, 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4127 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436718+0000", 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436719+0000", 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9999999999999995e-07 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436724+0000", 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7099999999999998e-06 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436739+0000", 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.482e-05 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436744+0000", 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.775e-06 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436791+0000", 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7420000000000003e-05 2024-09-16T14:07:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.436886+0000", 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4942e-05 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.437094+0000", 2024-09-16T14:07:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020822399999999999 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.439088+0000", 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019939340000000002 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.441989+0000", 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029012360000000002 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.442047+0000", 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.442148+0000", 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010031600000000001 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:4161 3.2 3:467b8b19:::rbd_data.1136494ba971.000000000000005a:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.977702359999999, 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054131559999999997, 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 4161 2024-09-16T14:07:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454230+0000", 2024-09-16T14:07:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454232+0000", 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.756e-06 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454239+0000", 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9110000000000004e-06 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454259+0000", 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9749999999999999e-05 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454266+0000", 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8569999999999995e-06 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454334+0000", 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7104e-05 2024-09-16T14:07:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454500+0000", 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016668100000000001 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.454703+0000", 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020301299999999999 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.457949+0000", 2024-09-16T14:07:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032455309999999999 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459554+0000", 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016048 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459599+0000", 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5089999999999997e-05 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.459643+0000", 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4663000000000003e-05 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5304 3.e 3:73080399:::rbd_data.1136494ba971.0000000000000075:7 [read 114688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 16.71297281, 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052879110000000002, 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5304 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718960+0000", 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1800000000000001e-07 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718962+0000", 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.03e-06 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718969+0000", 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2490000000000004e-06 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.718983+0000", 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3145999999999999e-05 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724107+0000", 2024-09-16T14:07:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051242450000000004 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724185+0000", 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8077999999999993e-05 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.724248+0000", 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2744999999999996e-05 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9030 3.16 3:6f8d7c1d:::rbd_data.1136dd12cea0.0000000000000027:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.613493712, 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082377660000000005, 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9030 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818442+0000", 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7669999999999999e-06 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818452+0000", 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0763e-05 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818486+0000", 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3942000000000001e-05 2024-09-16T14:07:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818499+0000", 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2571e-05 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818602+0000", 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010291200000000001 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818781+0000", 2024-09-16T14:07:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017939099999999999 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.819167+0000", 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038538 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.822000+0000", 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028333490000000002 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826582+0000", 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045822989999999997 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826632+0000", 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9780000000000001e-05 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826677+0000", 2024-09-16T14:07:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4611999999999999e-05 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9432 3.5 3:a779edfa:::rbd_data.11362e0d5ebf.000000000000008f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:26.854698+0000", 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.577234637, 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053152340000000003, 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9432 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854698+0000", 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854698+0000", 2024-09-16T14:07:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854699+0000", 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.618e-06 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854708+0000", 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8559999999999996e-06 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854726+0000", 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7342000000000001e-05 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854735+0000", 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8559999999999996e-06 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854828+0000", 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3232000000000005e-05 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854949+0000", 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012077400000000001 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.855237+0000", 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028819400000000003 2024-09-16T14:07:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.856832+0000", 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001595015 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.859952+0000", 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031203849999999998 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.859991+0000", 2024-09-16T14:07:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8430000000000003e-05 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.860013+0000", 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2532e-05 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9433 3.5 3:a3d6a849:::rbd_data.11362e0d5ebf.0000000000000092:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:26.854753+0000", 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.577179743, 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057192140000000002, 2024-09-16T14:07:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9433 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854753+0000", 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854753+0000", 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854753+0000", 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9699999999999995e-07 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854758+0000", 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1160000000000001e-06 2024-09-16T14:07:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854770+0000", 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.222e-05 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.854776+0000", 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7769999999999999e-06 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.855664+0000", 2024-09-16T14:07:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088838700000000003 2024-09-16T14:07:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.855758+0000", 2024-09-16T14:07:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4233000000000004e-05 2024-09-16T14:07:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.856044+0000", 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028594900000000002 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.857128+0000", 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010842530000000001 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.860379+0000", 2024-09-16T14:07:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032508179999999999 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.860433+0000", 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3594999999999998e-05 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.860472+0000", 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9168999999999997e-05 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9465 3.5 3:a7744965:::rbd_object_map.11362e0d5ebf.000000000000000a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:26.863709+0000", 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.568223501, 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066737389999999997, 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9465 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863709+0000", 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863709+0000", 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863710+0000", 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0200000000000002e-07 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863713+0000", 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3160000000000001e-06 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863727+0000", 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3753e-05 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.863733+0000", 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1759999999999998e-06 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.864597+0000", 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086442099999999998 2024-09-16T14:07:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.864645+0000", 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7250000000000003e-05 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.864899+0000", 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025424900000000001 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.865641+0000", 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074190900000000001 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.870288+0000", 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046474639999999996 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.870340+0000", 2024-09-16T14:07:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2157000000000002e-05 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:26.870383+0000", 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2441999999999998e-05 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:9863 3.11 e39/23 3:8c69c14f:::rbd_data.1136a719415e.000000000000000f:head v 39'37, mlcod=37'34)", 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.039462133000001, 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081439630000000006, 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392472+0000", 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7239999999999999e-06 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395095+0000", 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002622943 2024-09-16T14:07:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395116+0000", 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0784e-05 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395127+0000", 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0838e-05 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395189+0000", 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2211000000000005e-05 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395228+0000", 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8695000000000001e-05 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.400576+0000", 2024-09-16T14:07:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053489289999999997 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.400614+0000", 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7839000000000001e-05 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13208 3.5 3:a0deb963:::rbd_data.11361fa8f1cb.000000000000016c:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.806582+0000", 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6253501259999998, 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067377100000000001, 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13208 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806582+0000", 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806582+0000", 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806585+0000", 2024-09-16T14:07:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806597+0000", 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2554e-05 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806639+0000", 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1427999999999997e-05 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806656+0000", 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6956000000000002e-05 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806759+0000", 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103812 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.807107+0000", 2024-09-16T14:07:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034792800000000001 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813320+0000", 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062127440000000001 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13209 3.5 3:a693b338:::rbd_data.11361fa8f1cb.000000000000016d:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6252418970000004, 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086662360000000008, 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13209 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806692+0000", 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.345e-06 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806699+0000", 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8399999999999997e-06 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806718+0000", 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8828e-05 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806729+0000", 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1069e-05 2024-09-16T14:07:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.807862+0000", 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011332429999999999 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.808093+0000", 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023125700000000001 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815357+0000", 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072636539999999996 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13221 3.5 3:a0deb963:::rbd_data.11361fa8f1cb.000000000000016c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.813642+0000", 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6182900340000002, 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062358709999999996, 2024-09-16T14:07:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13221 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813642+0000", 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813642+0000", 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813644+0000", 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2500000000000001e-06 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813651+0000", 2024-09-16T14:07:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1539999999999996e-06 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813669+0000", 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8547999999999999e-05 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813680+0000", 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0266e-05 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.813802+0000", 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121883 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.814023+0000", 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000221249 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.814574+0000", 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00055117899999999995 2024-09-16T14:07:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815599+0000", 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010245969999999999 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.819794+0000", 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004194985 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.819834+0000", 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0336999999999997e-05 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.819878+0000", 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4422999999999998e-05 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13226 3.5 3:a693b338:::rbd_data.11361fa8f1cb.000000000000016d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.815618+0000", 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.616314247, 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0067527730000000001, 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13226 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815618+0000", 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815618+0000", 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815620+0000", 2024-09-16T14:07:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.426e-06 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815629+0000", 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8380000000000008e-06 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815651+0000", 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2768e-05 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815661+0000", 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0013999999999999e-05 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815738+0000", 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7058000000000007e-05 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815836+0000", 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7843000000000005e-05 2024-09-16T14:07:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815994+0000", 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015771099999999999 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.816884+0000", 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088978199999999999 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.822285+0000", 2024-09-16T14:07:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005401004 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.822335+0000", 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0714999999999998e-05 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.822371+0000", 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5614000000000002e-05 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14523 3.e 3:76eab748:::rbd_data.11362defb50e.0000000000000178:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.726840+0000", 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.705092074, 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055500860000000001, 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14523 2024-09-16T14:07:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.726840+0000", 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.726840+0000", 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.726842+0000", 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.548e-06 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.726849+0000", 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5159999999999997e-06 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.726869+0000", 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9850000000000001e-05 2024-09-16T14:07:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.726877+0000", 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8890000000000002e-06 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.726935+0000", 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7942000000000002e-05 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.727094+0000", 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158608 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.727295+0000", 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020129100000000001 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.729460+0000", 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002165227 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.732301+0000", 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028403539999999998 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.732345+0000", 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4592999999999998e-05 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.732391+0000", 2024-09-16T14:07:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5268000000000001e-05 2024-09-16T14:07:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.558 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:39.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:39.561 INFO:teuthology.orchestra.run.smithi079.stdout:1651 writesame 0xdb2c2d8 thru 0xdb2efda data_size 0x17(0x2d03 bytes) 2024-09-16T14:07:39.561 INFO:teuthology.orchestra.run.smithi079.stdout:1652 read 0xc61b041 thru 0xc6279cd (0xc98d bytes) 2024-09-16T14:07:39.561 INFO:teuthology.orchestra.run.smithi079.stdout:1653 punch from 0x677df66 to 0x678494a, (0x69e4 bytes) 2024-09-16T14:07:39.561 INFO:teuthology.orchestra.run.smithi079.stdout:1655 read 0xa51e5a2 thru 0xa520368 (0x1dc7 bytes) 2024-09-16T14:07:39.561 INFO:teuthology.orchestra.run.smithi079.stdout:1657 write 0x4b203b2 thru 0x4b2a3a7 (0x9ff6 bytes) 2024-09-16T14:07:39.561 INFO:teuthology.orchestra.run.smithi079.stdout:1658 punch from 0x50934d7 to 0x5097165, (0x3c8e bytes) 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:1663 read 0x2b0b7d8 thru 0x2b0f9a0 (0x41c9 bytes) 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:1665 read 0xdff94b9 thru 0xe004bb4 (0xb6fc bytes) 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:1666 read 0x502ef28 thru 0x503ad1b (0xbdf4 bytes) 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:1670 read 0xa5ffac2 thru 0xa609ddb (0xa31a bytes) 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:1671 read 0x2541bc8 thru 0x254dece (0xc307 bytes) 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:1672 trunc from 0xe78348a to 0x9f6bd 2024-09-16T14:07:39.562 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:39.562 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:39.554+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:39.577 INFO:teuthology.orchestra.run.smithi079.stdout:1676 write 0x5c4b840 thru 0x5c4ceb9 (0x167a bytes) 2024-09-16T14:07:39.592 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:39.603 INFO:teuthology.orchestra.run.smithi079.stdout:1680 write 0x713d57f thru 0x7147c28 (0xa6aa bytes) 2024-09-16T14:07:39.644 INFO:teuthology.orchestra.run.smithi079.stdout:1681 write 0xa9ff899 thru 0xaa04df3 (0x555b bytes) 2024-09-16T14:07:39.660 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:39.654+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:39.669 INFO:teuthology.orchestra.run.smithi079.stdout:1682 write 0x98ddd3b thru 0x98e0cb8 (0x2f7e bytes) 2024-09-16T14:07:39.685 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:27073 3.a 3:52c0a4e1:::rbd_data.11367eb52311.0000000000000262:head [set-alloc-hint object_size 262144 write_size 262144,write 153915~1733 in=1733b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:39.682920+0000", 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0020575770000000001, 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002081458, 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 27073 2024-09-16T14:07:39.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.682920+0000", 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.682920+0000", 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.682921+0000", 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1680000000000001e-06 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.682929+0000", 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9219999999999997e-06 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.682945+0000", 2024-09-16T14:07:39.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5299999999999999e-05 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.682954+0000", 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9609999999999994e-06 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.683028+0000", 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4078000000000005e-05 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.683143+0000", 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115596 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.683241+0000", 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7751999999999995e-05 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:39.684328+0000", 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010874859999999999 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.690 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.691 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:39.691 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-16T14:07:39.691 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:39.693 INFO:teuthology.orchestra.run.smithi079.stdout:1683 writesame 0x11ac3f7 thru 0x11bc3c6 data_size 0x8(0xffd0 bytes) 2024-09-16T14:07:39.696 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:39.717 INFO:teuthology.orchestra.run.smithi079.stdout:1684 writesame 0x72ddfd8 thru 0x72e94bc data_size 0x1(0xb4e5 bytes) 2024-09-16T14:07:39.726 INFO:teuthology.orchestra.run.smithi079.stdout:1685 read 0x9db49a8 thru 0x9dc37e6 (0xee3f bytes) 2024-09-16T14:07:39.726 INFO:teuthology.orchestra.run.smithi079.stdout:1686 write 0x85b3ae thru 0x8612dc (0x5f2f bytes) 2024-09-16T14:07:39.752 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:07:39.753 INFO:teuthology.orchestra.run.smithi079.stdout:1687 write 0x9011e4d thru 0x901422a (0x23de bytes) 2024-09-16T14:07:39.761 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:39.754+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:39.779 INFO:teuthology.orchestra.run.smithi079.stdout:1688 write 0xc1d669 thru 0xc29d2f (0xc6c7 bytes) 2024-09-16T14:07:39.807 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:39.807 INFO:teuthology.orchestra.run.smithi079.stdout:1690 punch from 0x1938712 to 0x1940d95, (0x8683 bytes) 2024-09-16T14:07:39.807 INFO:teuthology.orchestra.run.smithi079.stdout:1691 write 0xa33ecb9 thru 0xa341b1d (0x2e65 bytes) 2024-09-16T14:07:39.826 INFO:teuthology.orchestra.run.smithi079.stdout:1692 write 0x1789c45 thru 0x1792c9a (0x9056 bytes) 2024-09-16T14:07:39.834 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:39.835 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:39.835 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:39.835 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:39.835 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:39.845 INFO:teuthology.orchestra.run.smithi079.stdout:1694 read 0x1e08a9f thru 0x1e0e5d0 (0x5b32 bytes) 2024-09-16T14:07:39.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:39.857 INFO:teuthology.orchestra.run.smithi079.stdout:1695 write 0xd8ab859 thru 0xd8b2e59 (0x7601 bytes) 2024-09-16T14:07:39.880 INFO:teuthology.orchestra.run.smithi079.stdout:1696 write 0x60f9e9c thru 0x610977e (0xf8e3 bytes) 2024-09-16T14:07:39.880 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:39.874+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:39.902 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:39.903 INFO:teuthology.orchestra.run.smithi079.stdout:1699 trunc from 0xd8b2e5a to 0xe9e0a26 2024-09-16T14:07:39.914 INFO:teuthology.orchestra.run.smithi079.stdout:1700 write 0x36800a4 thru 0x368e91f (0xe87c bytes) 2024-09-16T14:07:39.941 INFO:teuthology.orchestra.run.smithi079.stdout:1701 punch from 0x39bddba to 0x39c6760, (0x89a6 bytes) 2024-09-16T14:07:39.941 INFO:teuthology.orchestra.run.smithi079.stdout:1702 writesame 0x4e88a0f thru 0x4e8f090 data_size 0x1(0x6682 bytes) 2024-09-16T14:07:39.963 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:39.958+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:39.966 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:39.966 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:39.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:39.966 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:39.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 39.483790167999999, 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.095465191000000005, 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479100+0000", 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9840000000000001e-06 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479107+0000", 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7920000000000004e-06 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479120+0000", 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3195e-05 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479131+0000", 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1416999999999999e-05 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479210+0000", 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8223e-05 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479224+0000", 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.436e-05 2024-09-16T14:07:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570355+0000", 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091131223999999997 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570668+0000", 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031290000000000002 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570990+0000", 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032147600000000001 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574367+0000", 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033769680000000002 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574482+0000", 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115021 2024-09-16T14:07:39.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574528+0000", 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6035999999999998e-05 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574560+0000", 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2595000000000002e-05 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12319 3.1d 3:bbef0c8d:::rbd_data.11364aaed44f.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.0585466530000005, 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031934152, 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12319 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904340+0000", 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.742e-06 2024-09-16T14:07:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904350+0000", 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6719999999999999e-06 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904372+0000", 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1787999999999999e-05 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904381+0000", 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9099999999999994e-06 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904477+0000", 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6571999999999996e-05 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904631+0000", 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000153311 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.912219+0000", 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075880990000000001 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936131+0000", 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023912255 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936208+0000", 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7464000000000002e-05 2024-09-16T14:07:39.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936244+0000", 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5369e-05 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936273+0000", 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8969999999999999e-05 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12330 3.d 3:b20fa72b:::rbd_data.11364aaed44f.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.9840671790000002, 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.032907032000000003, 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12330 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.980 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978820+0000", 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9860000000000001e-06 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978831+0000", 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1457999999999999e-05 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978855+0000", 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3201e-05 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978864+0000", 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6299999999999993e-06 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978954+0000", 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9420999999999996e-05 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.979080+0000", 2024-09-16T14:07:39.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012617800000000001 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.986286+0000", 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072064429999999999 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011593+0000", 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025306443000000001 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011654+0000", 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0764999999999998e-05 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011696+0000", 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2330999999999998e-05 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011725+0000", 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9176e-05 2024-09-16T14:07:39.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15971 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 851968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178362+0000", 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7845234890000001, 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021997244999999999, 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15971 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178362+0000", 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178362+0000", 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178362+0000", 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3000000000000002e-07 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178363+0000", 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3239999999999999e-06 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178367+0000", 2024-09-16T14:07:39.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178369+0000", 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200267+0000", 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021897349 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200312+0000", 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5181e-05 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200359+0000", 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7042000000000001e-05 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.990 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15973 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178382+0000", 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7845036309999998, 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022094526999999999, 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15973 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178382+0000", 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178382+0000", 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178382+0000", 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3300000000000003e-07 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178383+0000", 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1880000000000001e-06 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178387+0000", 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178389+0000", 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4269999999999998e-06 2024-09-16T14:07:39.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200375+0000", 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021986061000000001 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200429+0000", 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3464000000000002e-05 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200476+0000", 2024-09-16T14:07:39.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7478000000000002e-05 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15974 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7844937669999998, 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022185745, 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:39.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15974 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:39.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178392+0000", 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178393+0000", 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.308e-06 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178396+0000", 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4209999999999999e-06 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178399+0000", 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3120000000000001e-06 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200491+0000", 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022092166999999999 2024-09-16T14:07:39.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200529+0000", 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8380999999999997e-05 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200577+0000", 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7868000000000003e-05 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:39.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15976 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7844743960000002, 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022266966999999999, 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15976 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:40.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178412+0000", 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178416+0000", 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4489999999999999e-06 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178418+0000", 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2900000000000001e-06 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200596+0000", 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022178192999999999 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200633+0000", 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6792999999999998e-05 2024-09-16T14:07:40.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200678+0000", 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4743999999999997e-05 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15977 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:40.004 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7844646320000002, 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022367227999999999, 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15977 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.84e-07 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178422+0000", 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.257e-06 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178426+0000", 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178429+0000", 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4200000000000001e-06 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200694+0000", 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022265367000000001 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200735+0000", 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0908000000000002e-05 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200788+0000", 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2787999999999997e-05 2024-09-16T14:07:40.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15978 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.784453987, 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022476471000000001, 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.009 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15978 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178432+0000", 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0800000000000001e-07 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178435+0000", 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7249999999999999e-06 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178439+0000", 2024-09-16T14:07:40.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.507e-06 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178441+0000", 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5579999999999999e-06 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200803+0000", 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022361322999999999 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200841+0000", 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8353999999999999e-05 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200908+0000", 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6696e-05 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15980 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7844298510000001, 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022615139999999999, 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15980 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178456+0000", 2024-09-16T14:07:40.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8299999999999998e-07 2024-09-16T14:07:40.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178457+0000", 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.271e-06 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178460+0000", 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.455e-06 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178463+0000", 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.306e-06 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200933+0000", 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022470054999999999 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201000+0000", 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7117000000000001e-05 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201071+0000", 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0653000000000003e-05 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15982 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7844103369999997, 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022793758000000001, 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15982 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:40.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0499999999999999e-07 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178479+0000", 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.433e-06 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178482+0000", 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.63e-06 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178485+0000", 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4789999999999999e-06 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201098+0000", 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022612818 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201185+0000", 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7886999999999994e-05 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201269+0000", 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3206000000000003e-05 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15983 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7843980080000001, 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022918754, 2024-09-16T14:07:40.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15983 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178488+0000", 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178489+0000", 2024-09-16T14:07:40.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.373e-06 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178492+0000", 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5609999999999999e-06 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178495+0000", 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3499999999999999e-06 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201297+0000", 2024-09-16T14:07:40.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022801920999999999 2024-09-16T14:07:40.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201345+0000", 2024-09-16T14:07:40.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8643000000000002e-05 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201406+0000", 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0571e-05 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15984 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:40.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7843827160000001, 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023018235000000001, 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15984 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:40.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178504+0000", 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.37e-06 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178508+0000", 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4879999999999999e-06 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178510+0000", 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3779999999999999e-06 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201424+0000", 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022913830999999999 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201461+0000", 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7002000000000002e-05 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201521+0000", 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9620999999999997e-05 2024-09-16T14:07:40.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15986 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7843401409999999, 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023085504, 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15986 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178546+0000", 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7199999999999999e-07 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178548+0000", 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.097e-06 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178554+0000", 2024-09-16T14:07:40.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2820000000000002e-06 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178558+0000", 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.878e-06 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201540+0000", 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022982318000000002 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201575+0000", 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5138999999999997e-05 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201631+0000", 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5217999999999999e-05 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15988 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.045 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7843060099999999, 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023229864999999999, 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15988 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:40.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178580+0000", 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1300000000000001e-07 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178582+0000", 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178589+0000", 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178594+0000", 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1459999999999998e-06 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201654+0000", 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023060664000000002 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201725+0000", 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0451999999999996e-05 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201809+0000", 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4395999999999996e-05 2024-09-16T14:07:40.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15989 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.784287054, 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023391981999999999, 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15989 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178599+0000", 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178601+0000", 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178608+0000", 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9789999999999996e-06 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178612+0000", 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0890000000000002e-06 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201835+0000", 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023222702000000001 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201905+0000", 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0425000000000004e-05 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201990+0000", 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5062999999999995e-05 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15990 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7842688820000001, 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023564649, 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15990 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178617+0000", 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.63e-07 2024-09-16T14:07:40.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178619+0000", 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.097e-06 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178626+0000", 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6320000000000002e-06 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178629+0000", 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8480000000000003e-06 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202017+0000", 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023387811000000001 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202111+0000", 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4055999999999995e-05 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202181+0000", 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9741999999999997e-05 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15992 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7842286430000001, 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02370862, 2024-09-16T14:07:40.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15992 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6600000000000002e-07 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178659+0000", 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3379999999999999e-06 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178666+0000", 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7050000000000002e-06 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178670+0000", 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6789999999999998e-06 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202206+0000", 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023536148999999999 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202248+0000", 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2539e-05 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202365+0000", 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011674400000000001 2024-09-16T14:07:40.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15993 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.7842113299999998, 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024088607000000001, 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15993 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999999e-07 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178677+0000", 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2180000000000002e-06 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178684+0000", 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0770000000000002e-06 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178694+0000", 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0241000000000001e-05 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202580+0000", 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023885937999999999 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202667+0000", 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7425999999999999e-05 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202763+0000", 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5235000000000006e-05 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15995 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:40.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.784126487, 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024187608999999999, 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15995 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:40.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2800000000000003e-07 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178761+0000", 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6810000000000001e-06 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178765+0000", 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4259999999999996e-06 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178768+0000", 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6740000000000001e-06 2024-09-16T14:07:40.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202791+0000", 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024023224999999999 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202865+0000", 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4206000000000001e-05 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:40.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202946+0000", 2024-09-16T14:07:40.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1069000000000006e-05 2024-09-16T14:07:40.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:40.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:40.073 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:40.074 INFO:teuthology.orchestra.run.smithi079.stdout:1705 writesame 0xd524f02 thru 0xd52523c data_size 0x1(0x33b bytes) 2024-09-16T14:07:40.074 INFO:teuthology.orchestra.run.smithi079.stdout:1706 punch from 0xd181b5d to 0xd187d8e, (0x6231 bytes) 2024-09-16T14:07:40.074 INFO:teuthology.orchestra.run.smithi079.stdout:1709 writesame 0x95b3ee0 thru 0x95c2bca data_size 0x45(0xeceb bytes) 2024-09-16T14:07:40.075 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:40.076 INFO:teuthology.orchestra.run.smithi079.stdout:1710 read 0x4357d5f thru 0x4365a05 (0xdca7 bytes) 2024-09-16T14:07:40.076 INFO:teuthology.orchestra.run.smithi079.stdout:1711 trunc from 0xe9e0a26 to 0xdaded9b 2024-09-16T14:07:40.076 INFO:teuthology.orchestra.run.smithi079.stdout:1712 write 0xc80b7bc thru 0xc81b640 (0xfe85 bytes) 2024-09-16T14:07:40.076 INFO:teuthology.orchestra.run.smithi079.stdout:1713 trunc from 0xdaded9b to 0xa9ab670 2024-09-16T14:07:40.076 INFO:teuthology.orchestra.run.smithi079.stdout:1714 clone 12 order 25 su 8388608 sc 15 2024-09-16T14:07:40.077 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:40.070+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:40.176 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:40.174+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:40.181 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:40.181 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:40.181 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:40.181 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:40.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:40.277 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:40.270+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:40.293 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:40.293 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:40.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:40.378 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:40.370+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:40.400 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:918 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.665374185000001, 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015774501, 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 918 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730965+0000", 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730967+0000", 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730971+0000", 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5080000000000001e-06 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730973+0000", 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.091e-06 2024-09-16T14:07:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.739992+0000", 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090191430000000003 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740162+0000", 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016968899999999999 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740834+0000", 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067256100000000001 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.744120+0000", 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032859370000000001 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746643+0000", 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025232050000000002 2024-09-16T14:07:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746700+0000", 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6251000000000001e-05 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746739+0000", 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9201e-05 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1141 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.059499378000002, 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014042171000000001, 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1141 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336840+0000", 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2600000000000002e-07 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336844+0000", 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7419999999999999e-06 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336850+0000", 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336855+0000", 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4939999999999997e-06 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347139+0000", 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010284331000000001 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347252+0000", 2024-09-16T14:07:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011294300000000001 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347714+0000", 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046169700000000001 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350720+0000", 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030062359999999998 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350774+0000", 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3415999999999998e-05 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350837+0000", 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3151999999999994e-05 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350881+0000", 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4591e-05 2024-09-16T14:07:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1142 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.059479128, 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016069268000000001, 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1142 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336860+0000", 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336866+0000", 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5600000000000001e-06 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336872+0000", 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9410000000000001e-06 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336877+0000", 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.172e-06 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348737+0000", 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011860239999999999 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348829+0000", 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1657999999999996e-05 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349140+0000", 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031145499999999998 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352798+0000", 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036581550000000002 2024-09-16T14:07:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352815+0000", 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6883000000000001e-05 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352876+0000", 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0871e-05 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352929+0000", 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2781000000000002e-05 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1143 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.059457728999998, 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018939362000000001, 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1143 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.44e-07 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336884+0000", 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336891+0000", 2024-09-16T14:07:40.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336895+0000", 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0820000000000001e-06 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349584+0000", 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012689283000000001 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349708+0000", 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123793 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350151+0000", 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044259900000000001 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352993+0000", 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028426979999999998 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355720+0000", 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002726642 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355762+0000", 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2116000000000001e-05 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355820+0000", 2024-09-16T14:07:40.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8118999999999997e-05 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.059439622999999, 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:07:40.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.430 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.059417226000001, 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:07:40.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:07:40.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.058810515000001, 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.437 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:07:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:07:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:07:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:07:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:07:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.058736188000001, 2024-09-16T14:07:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:07:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1150 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.051386268000002, 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017570243999999999, 2024-09-16T14:07:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1150 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344954+0000", 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4959999999999999e-06 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344964+0000", 2024-09-16T14:07:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8719999999999997e-06 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344980+0000", 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6622000000000001e-05 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344988+0000", 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6879999999999999e-06 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354864+0000", 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098759160000000002 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354938+0000", 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4179999999999998e-05 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355250+0000", 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000311973 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.361925+0000", 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066749859999999999 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362285+0000", 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036007999999999998 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362462+0000", 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017646200000000001 2024-09-16T14:07:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362523+0000", 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1151 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.050897517999999, 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017278714000000001, 2024-09-16T14:07:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1151 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345442+0000", 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.42e-06 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345450+0000", 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2420000000000003e-06 2024-09-16T14:07:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345461+0000", 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1395e-05 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345467+0000", 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3600000000000001e-06 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355848+0000", 2024-09-16T14:07:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010380162 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355940+0000", 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2629999999999999e-05 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356352+0000", 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041188100000000001 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362037+0000", 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005684911 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362545+0000", 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050792899999999995 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362636+0000", 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1479999999999998e-05 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362720+0000", 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3304000000000001e-05 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1154 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.048516292999999, 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015133097999999999, 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1154 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347823+0000", 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347832+0000", 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347854+0000", 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2086000000000002e-05 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347864+0000", 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356827+0000", 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089628060000000002 2024-09-16T14:07:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356897+0000", 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0494000000000006e-05 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357171+0000", 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027356199999999998 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362741+0000", 2024-09-16T14:07:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055696920000000002 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362867+0000", 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012610799999999999 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362922+0000", 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4911000000000003e-05 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362955+0000", 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3810999999999999e-05 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1155 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.048229822, 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014975097999999999, 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1155 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348110+0000", 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6700000000000002e-07 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348114+0000", 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2560000000000004e-06 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348129+0000", 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4725e-05 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348134+0000", 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.152e-06 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357588+0000", 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094536949999999998 2024-09-16T14:07:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357666+0000", 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8664000000000004e-05 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357994+0000", 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032747700000000001 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362762+0000", 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047681829999999996 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363017+0000", 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025504500000000002 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363056+0000", 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9357000000000003e-05 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363084+0000", 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7577e-05 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1161 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.04525722, 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017258941999999999, 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1161 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351083+0000", 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2270000000000001e-06 2024-09-16T14:07:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351089+0000", 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.178e-06 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351108+0000", 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9457e-05 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351116+0000", 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6499999999999996e-06 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360527+0000", 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094112959999999996 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360645+0000", 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011791299999999999 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.361269+0000", 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062395199999999999 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365240+0000", 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039707470000000002 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368249+0000", 2024-09-16T14:07:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030092000000000001 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368302+0000", 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3146999999999998e-05 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368340+0000", 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8175e-05 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1162 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.043529476, 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017877857, 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1162 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352810+0000", 2024-09-16T14:07:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.234e-06 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352818+0000", 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352831+0000", 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2839e-05 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352838+0000", 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1790000000000002e-06 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363259+0000", 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010421616 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363370+0000", 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111134 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363713+0000", 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034221700000000002 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.367148+0000", 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034350029999999998 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370616+0000", 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034687419999999999 2024-09-16T14:07:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370660+0000", 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3365000000000002e-05 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370687+0000", 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7121999999999999e-05 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1164 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.042453451, 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016868919, 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1164 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353886+0000", 2024-09-16T14:07:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2130000000000001e-06 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353892+0000", 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1299999999999998e-06 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353903+0000", 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0821e-05 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353909+0000", 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2909999999999998e-06 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364201+0000", 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010292318 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364295+0000", 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3667999999999998e-05 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364657+0000", 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036279999999999998 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.367209+0000", 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025513250000000001 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370697+0000", 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034881600000000001 2024-09-16T14:07:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370729+0000", 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1977999999999998e-05 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370754+0000", 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5215000000000002e-05 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1166 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.040205867000001, 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014686309999999999, 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1166 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9400000000000004e-07 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356137+0000", 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.473e-06 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356161+0000", 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3968000000000002e-05 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356167+0000", 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5840000000000002e-06 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365282+0000", 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091153269999999995 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365369+0000", 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7064000000000005e-05 2024-09-16T14:07:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365709+0000", 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034009900000000001 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368388+0000", 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002679393 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370762+0000", 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023732319999999999 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370793+0000", 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1550000000000001e-05 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370819+0000", 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5826e-05 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1325 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.676597699999999, 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014200757, 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1325 2024-09-16T14:07:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719742+0000", 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3459999999999999e-06 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719750+0000", 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6569999999999997e-06 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719777+0000", 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7370000000000001e-05 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719785+0000", 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2190000000000004e-06 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728088+0000", 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083028860000000006 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728229+0000", 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014092799999999999 2024-09-16T14:07:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728704+0000", 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047427599999999998 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.729579+0000", 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00087516299999999998 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733865+0000", 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042860570000000002 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733907+0000", 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789999999999998e-05 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733942+0000", 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5064999999999997e-05 2024-09-16T14:07:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.087811395999999, 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:07:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:07:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.4598528149999996, 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:07:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:07:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.9296923489999998, 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:07:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:07:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:07:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.522 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:40.523 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:40.524 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:40.518+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:40.624 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:40.618+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:40.630 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:40.630 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:40.630 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:40.631 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:40.640 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:40.725 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:40.718+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:40.790 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:40.790 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:40.790 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:40.790 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:40.790 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:40.799 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:40.843 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:40.838+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:40.884 INFO:teuthology.orchestra.run.smithi079.stdout:flattening image image_client.0-clone11 2024-09-16T14:07:40.899 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9727 3.1a e39/23 3:5b31ba1a:::rbd_data.1136a719415e.000000000000000c:head v 39'121, mlcod=39'118)", 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.679330639, 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026296277, 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.900 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216595+0000", 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7570000000000001e-06 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227934+0000", 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011338568 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227959+0000", 2024-09-16T14:07:40.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5874e-05 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227969+0000", 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.397e-06 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.228059+0000", 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9798999999999999e-05 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.228131+0000", 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2268000000000007e-05 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.242760+0000", 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014629085999999999 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.242890+0000", 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012952800000000001 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12155 3.0 3:06ec4796:::rbd_data.11364aaed44f.0000000000000005:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.477533+0000", 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.418390517000001, 2024-09-16T14:07:40.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030057369, 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12155 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477533+0000", 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477533+0000", 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477536+0000", 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.92e-06 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477551+0000", 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4847e-05 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477586+0000", 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4663999999999999e-05 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477600+0000", 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4297000000000001e-05 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477686+0000", 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6111000000000003e-05 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477792+0000", 2024-09-16T14:07:40.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105863 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.482474+0000", 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046815679999999997 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.507376+0000", 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024902783000000001 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.507397+0000", 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0397000000000001e-05 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.507527+0000", 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000129791 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.507591+0000", 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4128000000000007e-05 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12183 3.0 3:06ec4796:::rbd_data.11364aaed44f.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.597948+0000", 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.297976075999999, 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021084459, 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12183 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597948+0000", 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597948+0000", 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597951+0000", 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8490000000000002e-06 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597965+0000", 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4313e-05 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597997+0000", 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2224000000000002e-05 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.598008+0000", 2024-09-16T14:07:40.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1024e-05 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.598076+0000", 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7966000000000006e-05 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.598158+0000", 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1779000000000004e-05 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.600850+0000", 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026922679999999998 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.618647+0000", 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017796816999999999 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.618773+0000", 2024-09-16T14:07:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012578900000000001 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.618926+0000", 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015294199999999999 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.619032+0000", 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106488 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26859 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426144+0000", 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4697796379999999, 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024786332000000001, 2024-09-16T14:07:40.919 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.920 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.920 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.920 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.920 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.920 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26859 2024-09-16T14:07:40.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426144+0000", 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426144+0000", 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426145+0000", 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1600000000000001e-07 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426148+0000", 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3550000000000001e-06 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426155+0000", 2024-09-16T14:07:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426160+0000", 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8770000000000001e-06 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.441822+0000", 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015661616 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.441932+0000", 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109998 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.442335+0000", 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040383599999999997 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.447001+0000", 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046651000000000001 2024-09-16T14:07:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450811+0000", 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038108830000000002 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450861+0000", 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9642999999999999e-05 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450931+0000", 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9528000000000002e-05 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26860 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426166+0000", 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4697576720000001, 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024875105000000002, 2024-09-16T14:07:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26860 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426166+0000", 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426166+0000", 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426167+0000", 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9999999999999997e-07 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426170+0000", 2024-09-16T14:07:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4240000000000002e-06 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426177+0000", 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5729999999999996e-06 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426182+0000", 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6480000000000002e-06 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.443169+0000", 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016987825000000002 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.443300+0000", 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013106700000000001 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.443866+0000", 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056600400000000005 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450801+0000", 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069350749999999997 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450944+0000", 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014229600000000001 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450985+0000", 2024-09-16T14:07:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0788000000000003e-05 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451041+0000", 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6805000000000002e-05 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26861 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426186+0000", 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.469737853, 2024-09-16T14:07:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025111023, 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26861 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426186+0000", 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426186+0000", 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426187+0000", 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0500000000000003e-07 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426190+0000", 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3579999999999999e-06 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426198+0000", 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7109999999999995e-06 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426202+0000", 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4220000000000002e-06 2024-09-16T14:07:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.445010+0000", 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018807395000000001 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.445278+0000", 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026810199999999999 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.445987+0000", 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000709269 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451052+0000", 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050649320000000003 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451138+0000", 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6420000000000003e-05 2024-09-16T14:07:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451228+0000", 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9659000000000003e-05 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451297+0000", 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.915e-05 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26862 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426207+0000", 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4697172700000001, 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025300552, 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26862 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426207+0000", 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426207+0000", 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426207+0000", 2024-09-16T14:07:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0500000000000003e-07 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426211+0000", 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6330000000000002e-06 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426217+0000", 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5200000000000003e-06 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426222+0000", 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4129999999999999e-06 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.447065+0000", 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020843413000000002 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.447158+0000", 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2799000000000004e-05 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.447745+0000", 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058742899999999999 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451391+0000", 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003645456 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451403+0000", 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1788e-05 2024-09-16T14:07:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451446+0000", 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3677000000000002e-05 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451507+0000", 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0819000000000002e-05 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26863 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426227+0000", 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4696972100000001, 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026712219999999998, 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26863 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426227+0000", 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426227+0000", 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426227+0000", 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8800000000000002e-07 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426231+0000", 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426238+0000", 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5780000000000004e-06 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426242+0000", 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4479999999999996e-06 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.448572+0000", 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022329664999999999 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.448856+0000", 2024-09-16T14:07:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000284006 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.449757+0000", 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000900897 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.452837+0000", 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030808739999999999 2024-09-16T14:07:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.452848+0000", 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0807e-05 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.452900+0000", 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1554999999999997e-05 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.452939+0000", 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9153000000000003e-05 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26885 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.456520+0000", 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4394042229999999, 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023841016, 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26885 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456520+0000", 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456520+0000", 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456520+0000", 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4099999999999998e-07 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456524+0000", 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7639999999999999e-06 2024-09-16T14:07:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456529+0000", 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3369999999999999e-06 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456534+0000", 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2819999999999998e-06 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.474117+0000", 2024-09-16T14:07:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017583016999999999 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.474214+0000", 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7270999999999998e-05 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.474630+0000", 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041571700000000002 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.476011+0000", 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013808539999999999 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.480275+0000", 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042646400000000001 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.480323+0000", 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7574000000000002e-05 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.480361+0000", 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7919000000000001e-05 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27014 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.602326+0000", 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.2935981220000001, 2024-09-16T14:07:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022684165999999999, 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27014 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602326+0000", 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602326+0000", 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602326+0000", 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602330+0000", 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8539999999999999e-06 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602338+0000", 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4800000000000004e-06 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602344+0000", 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.618696+0000", 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016351853999999999 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.618751+0000", 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.503e-05 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.618972+0000", 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022084200000000001 2024-09-16T14:07:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.621202+0000", 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002230246 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.624900+0000", 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036982180000000001 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.624959+0000", 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9175000000000002e-05 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.625010+0000", 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0575000000000003e-05 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27043 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.643108+0000", 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.252815751, 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022120593000000001, 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27043 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643108+0000", 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643108+0000", 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643109+0000", 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.75e-07 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643113+0000", 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0709999999999996e-06 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643122+0000", 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8119999999999997e-06 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643127+0000", 2024-09-16T14:07:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3800000000000002e-06 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.659795+0000", 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016667996000000001 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.659914+0000", 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118627 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.660734+0000", 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082016999999999995 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665132+0000", 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043983759999999998 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665141+0000", 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3809999999999999e-06 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665200+0000", 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9382999999999997e-05 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665229+0000", 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8722e-05 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27044 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.643134+0000", 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.25279023, 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022477451999999998, 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27044 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643134+0000", 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643134+0000", 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643134+0000", 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6199999999999997e-07 2024-09-16T14:07:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643146+0000", 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1365999999999999e-05 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643164+0000", 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8479999999999999e-05 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643171+0000", 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7769999999999997e-06 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.662090+0000", 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018918852999999999 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.662179+0000", 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8844000000000005e-05 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.662638+0000", 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00045959000000000001 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665246+0000", 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026078239999999999 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665430+0000", 2024-09-16T14:07:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018435899999999999 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665546+0000", 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011579200000000001 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665611+0000", 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4905000000000003e-05 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27045 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.643177+0000", 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.252747367, 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023067443, 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27045 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643177+0000", 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643177+0000", 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643177+0000", 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2500000000000005e-07 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643181+0000", 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.98e-06 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643191+0000", 2024-09-16T14:07:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.312e-06 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643195+0000", 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8860000000000003e-06 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.663428+0000", 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020232624000000001 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.663574+0000", 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014559999999999999 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.664163+0000", 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058896000000000003 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665638+0000", 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014754880000000001 2024-09-16T14:07:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.666019+0000", 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038109299999999999 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.666115+0000", 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6114999999999997e-05 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.666244+0000", 2024-09-16T14:07:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012866 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27164 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.779097+0000", 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1168265319999999, 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023825177999999999, 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27164 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779097+0000", 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779097+0000", 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779098+0000", 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7800000000000002e-07 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779101+0000", 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9280000000000002e-06 2024-09-16T14:07:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779108+0000", 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3379999999999999e-06 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779112+0000", 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.442e-06 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.796157+0000", 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017044626 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.796293+0000", 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013614000000000001 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.796800+0000", 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050668700000000002 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.799036+0000", 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022356569999999998 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.802849+0000", 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038136469999999999 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.802891+0000", 2024-09-16T14:07:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1646000000000001e-05 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.802923+0000", 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1689000000000001e-05 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27291 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.913623+0000", 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.98230127899999997, 2024-09-16T14:07:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022489499999999999, 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27291 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913623+0000", 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913623+0000", 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913623+0000", 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8200000000000001e-07 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913625+0000", 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913628+0000", 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.455e-06 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913631+0000", 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5040000000000001e-06 2024-09-16T14:07:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.931246+0000", 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017615071 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.931309+0000", 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2737999999999995e-05 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.931580+0000", 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027167800000000002 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936026+0000", 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044450779999999999 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936032+0000", 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1829999999999999e-06 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936082+0000", 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0331999999999999e-05 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936112+0000", 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0099000000000001e-05 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27292 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.913633+0000", 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.98229071300000004, 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023172521000000001, 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27292 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913633+0000", 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913633+0000", 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913634+0000", 2024-09-16T14:07:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8999999999999998e-07 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913636+0000", 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0829999999999998e-06 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913639+0000", 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.433e-06 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913641+0000", 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.932658+0000", 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019016043999999999 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.932772+0000", 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011435800000000001 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.933190+0000", 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041830300000000002 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936125+0000", 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029344639999999999 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936737+0000", 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00061256699999999995 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936782+0000", 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5082e-05 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936806+0000", 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3431e-05 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27293 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.913644+0000", 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.98227984199999996, 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023507556999999998, 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27293 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913644+0000", 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913644+0000", 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913644+0000", 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8700000000000002e-07 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913647+0000", 2024-09-16T14:07:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2060000000000001e-06 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913650+0000", 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913653+0000", 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6910000000000002e-06 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.934466+0000", 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020813420999999999 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.934564+0000", 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7666999999999998e-05 2024-09-16T14:07:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.935003+0000", 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043941999999999999 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936413+0000", 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014100079999999999 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.937102+0000", 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00068838600000000003 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.937130+0000", 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8826000000000001e-05 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.937152+0000", 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1229999999999998e-05 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27315 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.940451+0000", 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.95547330900000005, 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023080931999999998, 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27315 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940451+0000", 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940451+0000", 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940451+0000", 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9400000000000001e-07 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940454+0000", 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940457+0000", 2024-09-16T14:07:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6399999999999999e-06 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940462+0000", 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1280000000000001e-06 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.957003+0000", 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01654131 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.957112+0000", 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108721 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.957609+0000", 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000497858 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.958664+0000", 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001054269 2024-09-16T14:07:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.963434+0000", 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047702530000000003 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.963485+0000", 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1385999999999999e-05 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.963532+0000", 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6257000000000001e-05 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27371 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:40.007085+0000", 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.88883852200000002, 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021070558, 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27371 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007085+0000", 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007085+0000", 2024-09-16T14:07:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007086+0000", 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9999999999999996e-07 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007091+0000", 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1889999999999998e-06 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007102+0000", 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0954e-05 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007118+0000", 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5586e-05 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.023202+0000", 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016083745999999999 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.023382+0000", 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000179872 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.023899+0000", 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051745299999999997 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.027682+0000", 2024-09-16T14:07:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037834399999999999 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028062+0000", 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00037993199999999998 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028106+0000", 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3430000000000003e-05 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028156+0000", 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0155999999999999e-05 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27372 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:40.007127+0000", 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.88879666400000001, 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021560835, 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27372 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007127+0000", 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007127+0000", 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007128+0000", 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8899999999999998e-07 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007141+0000", 2024-09-16T14:07:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3042e-05 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007157+0000", 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6297999999999999e-05 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007168+0000", 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0475999999999999e-05 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.025119+0000", 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017950679000000001 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.025282+0000", 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016368399999999999 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.025690+0000", 2024-09-16T14:07:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040757400000000001 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028269+0000", 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025796209999999998 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028569+0000", 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029910599999999999 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028628+0000", 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9913e-05 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028688+0000", 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9653e-05 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.026 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:41.028 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:41.029 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:41.022+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:41.060 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-16T14:07:41.129 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:41.122+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:41.134 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:41.134 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:41.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:41.230 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:41.222+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:41.289 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:41.289 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:41.290 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:41.290 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:41.291 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:41.291 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:41.291 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:41.293 INFO:teuthology.orchestra.run.smithi079.stdout:1719 trunc from 0xa9ab670 to 0xd976084 2024-09-16T14:07:41.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:41.331 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:41.326+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:41.332 INFO:teuthology.orchestra.run.smithi079.stdout:1720 trunc from 0xd976084 to 0x21c8656 2024-09-16T14:07:41.388 INFO:teuthology.orchestra.run.smithi079.stdout:1721 write 0x4b49791 thru 0x4b5393c (0xa1ac bytes) 2024-09-16T14:07:41.394 INFO:teuthology.orchestra.run.smithi079.stdout:1722 read 0x1bf7ed2 thru 0x1bfd645 (0x5774 bytes) 2024-09-16T14:07:41.395 INFO:teuthology.orchestra.run.smithi079.stdout:1723 read 0x2f8ecd6 thru 0x2f959b1 (0x6cdc bytes) 2024-09-16T14:07:41.405 INFO:teuthology.orchestra.run.smithi079.stdout:1724 write 0x5353027 thru 0x5360f5d (0xdf37 bytes) 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9779 3.7 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.115083238, 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042761995999999997, 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.413 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9779 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293651+0000", 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.021e-06 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305709+0000", 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012058627000000001 2024-09-16T14:07:41.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305754+0000", 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4137000000000001e-05 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305769+0000", 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5257e-05 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305971+0000", 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000201833 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.306248+0000", 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027726999999999998 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.307018+0000", 2024-09-16T14:07:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076971799999999996 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.329087+0000", 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022069787 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336319+0000", 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072311789999999999 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336375+0000", 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6406999999999997e-05 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336410+0000", 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4759999999999999e-05 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12172 3.12 3:4dae129f:::rbd_data.11364aaed44f.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.86178795, 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031007818999999999, 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12172 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.420 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546946+0000", 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2270000000000002e-06 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546958+0000", 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.173e-05 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546989+0000", 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1449000000000003e-05 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547001+0000", 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1216999999999999e-05 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547125+0000", 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012428299999999999 2024-09-16T14:07:41.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547345+0000", 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022038799999999999 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.555417+0000", 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080718119999999994 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577743+0000", 2024-09-16T14:07:41.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02232599 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577765+0000", 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1675000000000002e-05 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577879+0000", 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011386400000000001 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577951+0000", 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2183999999999999e-05 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14144 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620414+0000", 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7883173709999998, 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022268178999999999, 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14144 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620414+0000", 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620414+0000", 2024-09-16T14:07:41.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620414+0000", 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1099999999999996e-07 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620416+0000", 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1229999999999998e-06 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620422+0000", 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620426+0000", 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6959999999999998e-06 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642494+0000", 2024-09-16T14:07:41.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022068641999999999 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642573+0000", 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8341000000000001e-05 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642682+0000", 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108995 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14146 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620432+0000", 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7882987799999999, 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022436186, 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14146 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620432+0000", 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620432+0000", 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620433+0000", 2024-09-16T14:07:41.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620436+0000", 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9909999999999998e-06 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620442+0000", 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0000000000000002e-06 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620446+0000", 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8039999999999999e-06 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642723+0000", 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022277267 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642781+0000", 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8094000000000003e-05 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642868+0000", 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7507000000000001e-05 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14155 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620449+0000", 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7882816410000002, 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022645345000000001, 2024-09-16T14:07:41.435 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14155 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620449+0000", 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620449+0000", 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620450+0000", 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999999999999997e-07 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620452+0000", 2024-09-16T14:07:41.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7430000000000001e-06 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620459+0000", 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3039999999999999e-06 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620463+0000", 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3719999999999998e-06 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642911+0000", 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022447501000000002 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642992+0000", 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0894000000000002e-05 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643095+0000", 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103041 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.439 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14158 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620469+0000", 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7882616989999995, 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023038100999999998, 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.440 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14158 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620469+0000", 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620469+0000", 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620470+0000", 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.89e-07 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620472+0000", 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.906e-06 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620478+0000", 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8830000000000003e-06 2024-09-16T14:07:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620481+0000", 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7929999999999999e-06 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643137+0000", 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022655589 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643251+0000", 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114231 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643507+0000", 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025620999999999999 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14159 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620485+0000", 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7882458259999998, 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023253517000000001, 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14159 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620485+0000", 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620485+0000", 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620486+0000", 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9999999999999997e-07 2024-09-16T14:07:41.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620499+0000", 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2818e-05 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620505+0000", 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2500000000000003e-06 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620509+0000", 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643534+0000", 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023024747000000002 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643620+0000", 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6522999999999998e-05 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643739+0000", 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118468 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14162 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620535+0000", 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.788196374, 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023321043999999999, 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.449 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14162 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620535+0000", 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620535+0000", 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620536+0000", 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9300000000000006e-07 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620538+0000", 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1600000000000001e-06 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620545+0000", 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7850000000000003e-06 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620552+0000", 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643766+0000", 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023214067000000001 2024-09-16T14:07:41.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643809+0000", 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3396999999999998e-05 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643856+0000", 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6628999999999997e-05 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14163 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620556+0000", 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7881753830000005, 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023398592999999999, 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14163 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620556+0000", 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620556+0000", 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620556+0000", 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5499999999999997e-07 2024-09-16T14:07:41.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620558+0000", 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620564+0000", 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9719999999999997e-06 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620568+0000", 2024-09-16T14:07:41.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8890000000000004e-06 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643872+0000", 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023303707999999999 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643909+0000", 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7219999999999999e-05 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643954+0000", 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4982999999999999e-05 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14165 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620815+0000", 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7879158759999996, 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023234923000000001, 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.458 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14165 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620815+0000", 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620815+0000", 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620816+0000", 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2529999999999999e-06 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620820+0000", 2024-09-16T14:07:41.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5030000000000002e-06 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620829+0000", 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3219999999999999e-06 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620834+0000", 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8930000000000004e-06 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643970+0000", 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023135572 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644005+0000", 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5296999999999998e-05 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644050+0000", 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5083000000000002e-05 2024-09-16T14:07:41.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14169 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620858+0000", 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.787873115, 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023309318999999998, 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14169 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620858+0000", 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620858+0000", 2024-09-16T14:07:41.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620858+0000", 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.13e-07 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620861+0000", 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3460000000000001e-06 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620868+0000", 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620872+0000", 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0890000000000002e-06 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644065+0000", 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023192023999999999 2024-09-16T14:07:41.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644108+0000", 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3302e-05 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644167+0000", 2024-09-16T14:07:41.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9398000000000003e-05 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14170 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7878548089999997, 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023454853000000001, 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14170 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620877+0000", 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1500000000000005e-07 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620879+0000", 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0269999999999998e-06 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620884+0000", 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7180000000000001e-06 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620891+0000", 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2319999999999998e-06 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644192+0000", 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023301776 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644255+0000", 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2948999999999996e-05 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644331+0000", 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5636000000000006e-05 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14172 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7878367800000001, 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023574827999999999, 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14172 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620895+0000", 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4300000000000002e-07 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620897+0000", 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.939e-06 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620903+0000", 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7450000000000001e-06 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620908+0000", 2024-09-16T14:07:41.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7769999999999999e-06 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644351+0000", 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023442411 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644388+0000", 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7441999999999998e-05 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644469+0000", 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0771000000000003e-05 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14175 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7878161969999997, 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023718466000000001, 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14175 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620927+0000", 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2113000000000001e-05 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620943+0000", 2024-09-16T14:07:41.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620947+0000", 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5669999999999996e-06 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644494+0000", 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023546886999999999 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644563+0000", 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8485000000000006e-05 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644633+0000", 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0529000000000002e-05 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.481 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14176 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7877790510000002, 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023877883999999999, 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14176 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9399999999999995e-07 2024-09-16T14:07:41.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620954+0000", 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0200000000000001e-06 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620960+0000", 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8440000000000003e-06 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620964+0000", 2024-09-16T14:07:41.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7459999999999998e-06 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644666+0000", 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023702375000000001 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644720+0000", 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3813000000000001e-05 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644830+0000", 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109592 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14179 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7877595460000002, 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024008551999999999, 2024-09-16T14:07:41.486 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14179 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620972+0000", 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000003e-07 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620981+0000", 2024-09-16T14:07:41.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4420000000000004e-06 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620988+0000", 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.297e-06 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620992+0000", 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8979999999999998e-06 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644865+0000", 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023872899999999999 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644910+0000", 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5794999999999998e-05 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644980+0000", 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9708000000000004e-05 2024-09-16T14:07:41.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14180 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7877354030000001, 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024190943999999999, 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.491 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14180 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0100000000000005e-07 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620998+0000", 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9829999999999999e-06 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.621004+0000", 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8270000000000003e-06 2024-09-16T14:07:41.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.621009+0000", 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4530000000000001e-06 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645013+0000", 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024003295000000001 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645077+0000", 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4139000000000004e-05 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645187+0000", 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109746 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:17276 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 3956736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.495 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:36.251735+0000", 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.1569958119999999, 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022215209999999999, 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 17276 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251735+0000", 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251735+0000", 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251736+0000", 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3200000000000004e-07 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251738+0000", 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8920000000000001e-06 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251744+0000", 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9780000000000002e-06 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251749+0000", 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4979999999999997e-06 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.273767+0000", 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022017256999999998 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.273855+0000", 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8691999999999997e-05 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.273950+0000", 2024-09-16T14:07:41.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5161000000000006e-05 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:17277 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 4038656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:36.251755+0000", 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.1569764400000002, 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022353208999999999, 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 17277 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251755+0000", 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251755+0000", 2024-09-16T14:07:41.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251755+0000", 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.06e-07 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251758+0000", 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0319999999999999e-06 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251765+0000", 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7039999999999997e-06 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251768+0000", 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4240000000000002e-06 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.273971+0000", 2024-09-16T14:07:41.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022202403999999999 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274030+0000", 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9750000000000002e-05 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274108+0000", 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7288999999999998e-05 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:17278 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 3629056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:36.251773+0000", 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.1569575920000004, 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022542092, 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 17278 2024-09-16T14:07:41.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251773+0000", 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251773+0000", 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251774+0000", 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.46e-07 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251776+0000", 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.88e-06 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:41.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251781+0000", 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1039999999999998e-06 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251785+0000", 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5159999999999996e-06 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274146+0000", 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022360604999999999 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274226+0000", 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9837000000000001e-05 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274316+0000", 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9703999999999994e-05 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:41.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:41.510 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:41.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:41.512 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:1727 read 0x28f1015 thru 0x28fada5 (0x9d91 bytes) 2024-09-16T14:07:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:1728 write 0x9d444ba thru 0x9d4b1e9 (0x6d30 bytes) 2024-09-16T14:07:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:1729 read 0x1068057 thru 0x1077fb4 (0xff5e bytes) 2024-09-16T14:07:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:1730 write 0x3075594 thru 0x3078363 (0x2dd0 bytes) 2024-09-16T14:07:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:1731 writesame 0xb53848 thru 0xb549f3 data_size 0x6(0x11ac bytes) 2024-09-16T14:07:41.514 INFO:teuthology.orchestra.run.smithi079.stdout:1732 punch from 0x23e7c06 to 0x23e7ff7, (0x3f1 bytes) 2024-09-16T14:07:41.514 INFO:teuthology.orchestra.run.smithi079.stdout:1734 punch from 0x12c50e2 to 0x12d2b66, (0xda84 bytes) 2024-09-16T14:07:41.514 INFO:teuthology.orchestra.run.smithi079.stdout:1735 write 0x30a3a50 thru 0x30a4c43 (0x11f4 bytes) 2024-09-16T14:07:41.514 INFO:teuthology.orchestra.run.smithi079.stdout:1736 writesame 0xa3e80e3 thru 0xa3ee090 data_size 0x25(0x5fae bytes) 2024-09-16T14:07:41.514 INFO:teuthology.orchestra.run.smithi079.stdout:1738 write 0xecfc0ab thru 0xecfd35f (0x12b5 bytes) 2024-09-16T14:07:41.514 INFO:teuthology.orchestra.run.smithi079.stdout:1739 read 0x33e11f4 thru 0x33e78dc (0x66e9 bytes) 2024-09-16T14:07:41.514 INFO:teuthology.orchestra.run.smithi079.stdout:1740 write 0x12a59cd thru 0x12b5095 (0xf6c9 bytes) 2024-09-16T14:07:41.514 INFO:teuthology.orchestra.run.smithi079.stdout:1741 write 0xc205353 thru 0xc213f97 (0xec45 bytes) 2024-09-16T14:07:41.514 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:41.506+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:41.515 INFO:teuthology.orchestra.run.smithi079.stdout:1742 trunc from 0xecfd360 to 0xbb2d94b 2024-09-16T14:07:41.530 INFO:teuthology.orchestra.run.smithi079.stdout:1743 read 0x16f28b0 thru 0x1701887 (0xefd8 bytes) 2024-09-16T14:07:41.531 INFO:teuthology.orchestra.run.smithi079.stdout:1745 write 0xa9fa83a thru 0xaa01323 (0x6aea bytes) 2024-09-16T14:07:41.536 INFO:teuthology.orchestra.run.smithi079.stdout:1746 read 0xb4f8727 thru 0xb4fcbc2 (0x449c bytes) 2024-09-16T14:07:41.536 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:41.536 INFO:teuthology.orchestra.run.smithi079.stdout:1748 write 0x5edadd6 thru 0x5ee38bf (0x8aea bytes) 2024-09-16T14:07:41.564 INFO:teuthology.orchestra.run.smithi079.stdout:1749 trunc from 0xbb2d94b to 0x1b4344d 2024-09-16T14:07:41.594 INFO:teuthology.orchestra.run.smithi079.stdout:1752 read 0x1b113d0 thru 0x1b13391 (0x1fc2 bytes) 2024-09-16T14:07:41.595 INFO:teuthology.orchestra.run.smithi079.stdout:1753 punch from 0x17701e1 to 0x177daea, (0xd909 bytes) 2024-09-16T14:07:41.609 INFO:teuthology.orchestra.run.smithi079.stdout:1755 write 0x43caf7f thru 0x43d1db5 (0x6e37 bytes) 2024-09-16T14:07:41.612 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:41.606+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:41.617 INFO:teuthology.orchestra.run.smithi079.stdout:1757 trunc from 0x43d1db6 to 0x308a971 2024-09-16T14:07:41.628 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:41.628 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:41.637 INFO:teuthology.orchestra.run.smithi079.stdout:1759 punch from 0x27b9793 to 0x27c8681, (0xeeee bytes) 2024-09-16T14:07:41.637 INFO:teuthology.orchestra.run.smithi079.stdout:1760 writesame 0x249a198 thru 0x249c69e data_size 0x1(0x2507 bytes) 2024-09-16T14:07:41.638 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:41.647 INFO:teuthology.orchestra.run.smithi079.stdout:1761 read 0x1ff8409 thru 0x1ffd96c (0x5564 bytes) 2024-09-16T14:07:41.649 INFO:teuthology.orchestra.run.smithi079.stdout:1762 trunc from 0x308a971 to 0x43836ae 2024-09-16T14:07:41.657 INFO:teuthology.orchestra.run.smithi079.stdout:1763 read 0x85c6ab thru 0x85d613 (0xf69 bytes) 2024-09-16T14:07:41.671 INFO:teuthology.orchestra.run.smithi079.stdout:1764 write 0x5ecbc8c thru 0x5ed064a (0x49bf bytes) 2024-09-16T14:07:41.671 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:41.677 INFO:teuthology.orchestra.run.smithi079.stdout:1766 read 0x5448382 thru 0x54493e0 (0x105f bytes) 2024-09-16T14:07:41.692 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:41.694 INFO:teuthology.orchestra.run.smithi079.stdout:1768 write 0x941489c thru 0x9419f77 (0x56dc bytes) 2024-09-16T14:07:41.697 INFO:teuthology.orchestra.run.smithi079.stdout:1769 punch from 0x40cd147 to 0x40d8147, (0xb000 bytes) 2024-09-16T14:07:41.697 INFO:teuthology.orchestra.run.smithi079.stdout:1770 read 0x1d450b8 thru 0x1d483e7 (0x3330 bytes) 2024-09-16T14:07:41.698 INFO:teuthology.orchestra.run.smithi079.stdout:1771 read 0x5525f8e thru 0x5528b4d (0x2bc0 bytes) 2024-09-16T14:07:41.698 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:41.698 INFO:teuthology.orchestra.run.smithi079.stdout:1776 write 0x65f03e8 thru 0x65f2258 (0x1e71 bytes) 2024-09-16T14:07:41.705 INFO:teuthology.orchestra.run.smithi079.stdout:1777 read 0x7c7f35f thru 0x7c85b56 (0x67f8 bytes) 2024-09-16T14:07:41.708 INFO:teuthology.orchestra.run.smithi079.stdout:1778 trunc from 0x9419f78 to 0x7cc6f4e 2024-09-16T14:07:41.713 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:41.706+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:41.715 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:41.735 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:41.739 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:41.739 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:41.739 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:41.739 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:41.740 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:41.746 INFO:teuthology.orchestra.run.smithi079.stdout:1780 write 0xea59832 thru 0xea599e8 (0x1b7 bytes) 2024-09-16T14:07:41.748 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:41.753 INFO:teuthology.orchestra.run.smithi079.stdout:1781 write 0xdf2645f thru 0xdf28915 (0x24b7 bytes) 2024-09-16T14:07:41.755 INFO:teuthology.orchestra.run.smithi079.stdout:1782 read 0x244003f thru 0x2447083 (0x7045 bytes) 2024-09-16T14:07:41.756 INFO:teuthology.orchestra.run.smithi079.stdout:1783 read 0x15c836f thru 0x15d126f (0x8f01 bytes) 2024-09-16T14:07:41.757 INFO:teuthology.orchestra.run.smithi079.stdout:1784 read 0x15718c4 thru 0x1576340 (0x4a7d bytes) 2024-09-16T14:07:41.758 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:41.761 INFO:teuthology.orchestra.run.smithi079.stdout:1785 trunc from 0xea599e9 to 0xc63acc5 2024-09-16T14:07:41.767 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:07:41.776 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:41.776 INFO:teuthology.orchestra.run.smithi079.stdout:1789 read 0x3f4da15 thru 0x3f5b88d (0xde79 bytes) 2024-09-16T14:07:41.784 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:41.786 INFO:teuthology.orchestra.run.smithi079.stdout:1790 writesame 0xd638106 thru 0xd6417e5 data_size 0x47(0x96e0 bytes) 2024-09-16T14:07:41.788 INFO:teuthology.orchestra.run.smithi079.stdout:1791 writesame 0x70aec46 thru 0x70afcc3 data_size 0x1(0x107e bytes) 2024-09-16T14:07:41.793 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:41.793 INFO:teuthology.orchestra.run.smithi079.stdout:1793 writesame 0x28c46fc thru 0x28d4235 data_size 0x51(0xfb3a bytes) 2024-09-16T14:07:41.798 INFO:teuthology.orchestra.run.smithi079.stdout:1794 read 0x53a82d9 thru 0x53b6a56 (0xe77e bytes) 2024-09-16T14:07:41.798 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:41.798 INFO:teuthology.orchestra.run.smithi079.stdout:1796 read 0x13f9062 thru 0x13ffb3d (0x6adc bytes) 2024-09-16T14:07:41.800 INFO:teuthology.orchestra.run.smithi079.stdout:1797 trunc from 0xd6417e6 to 0xe0609d3 2024-09-16T14:07:41.807 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:41.811 INFO:teuthology.orchestra.run.smithi079.stdout:1798 clone 13 order 18 su 8192 sc 14 2024-09-16T14:07:41.813 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:41.806+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:41.828 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:41.829 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:41.829 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:41.830 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:41.848 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:41.848 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:41.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:41.915 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:41.906+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:41.985 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:41.985 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:41.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:42.015 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:42.010+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:42.093 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.093 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:42.146 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:42.142+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:42.198 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.198 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:42.217 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:42.210+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:42.302 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.302 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:42.317 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:42.310+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:42.410 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.411 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.418 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:42.410+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:42.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:42.518 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.518 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.519 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:42.514+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:42.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:42.619 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:42.614+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:42.624 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.624 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:42.720 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:42.714+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:42.733 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.733 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:42.786 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:07:42.821 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:42.814+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:42.854 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.854 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:42.879 INFO:teuthology.orchestra.run.smithi079.stdout:leaving image image_client.0-clone12 intact 2024-09-16T14:07:42.922 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:42.914+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:42.963 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:42.963 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:42.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:43.023 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:43.014+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:43.071 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-16T14:07:43.121 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:43.122 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:43.124 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:43.118+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:43.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:43.225 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:43.218+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:43.256 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:43.256 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:43.266 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:43.326 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:43.322+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:43.375 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:43.376 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:43.385 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:43.427 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:43.422+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:43.466 INFO:teuthology.orchestra.run.smithi079.stdout:1799 write 0x252f872 thru 0x2532ac6 (0x3255 bytes) 2024-09-16T14:07:43.478 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:43.478 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:43.487 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:43.523 INFO:teuthology.orchestra.run.smithi079.stdout:1800 trunc from 0xe0609d3 to 0x33e9c6 2024-09-16T14:07:43.529 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:43.522+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:43.592 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:43.593 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:43.602 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:43.629 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:43.622+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:43.678 INFO:teuthology.orchestra.run.smithi079.stdout:1801 write 0xd85db0e thru 0xd867ebd (0xa3b0 bytes) 2024-09-16T14:07:43.700 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:43.701 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:43.701 INFO:teuthology.orchestra.run.smithi079.stdout:1802 trunc from 0xd867ebe to 0x725cbb0 2024-09-16T14:07:43.710 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:43.718 INFO:teuthology.orchestra.run.smithi079.stdout:1803 write 0xf03ff0 thru 0xf108ae (0xc8bf bytes) 2024-09-16T14:07:43.738 INFO:teuthology.orchestra.run.smithi079.stdout:1804 trunc from 0x725cbb0 to 0x6a464ac 2024-09-16T14:07:43.751 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:43.746+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:43.764 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:07:43.764 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T14:07:43.768 INFO:teuthology.orchestra.run.smithi079.stdout:1807 writesame 0xc9cf995 thru 0xc9d31d8 data_size 0x1a(0x3844 bytes) 2024-09-16T14:07:43.768 INFO:teuthology.orchestra.run.smithi079.stdout:1808 read 0x4596984 thru 0x45a1e7d (0xb4fa bytes) 2024-09-16T14:07:43.768 INFO:teuthology.orchestra.run.smithi079.stdout:1810 writesame 0xc95e7d2 thru 0xc96cd0a data_size 0x7(0xe539 bytes) 2024-09-16T14:07:43.786 INFO:teuthology.orchestra.run.smithi079.stdout:1811 punch from 0x8680e28 to 0x8685ee4, (0x50bc bytes) 2024-09-16T14:07:43.786 INFO:teuthology.orchestra.run.smithi079.stdout:1812 write 0x60b8dfd thru 0x60c7b18 (0xed1c bytes) 2024-09-16T14:07:43.804 INFO:teuthology.orchestra.run.smithi079.stdout:1813 write 0x3b2f59a thru 0x3b35727 (0x618e bytes) 2024-09-16T14:07:43.816 INFO:teuthology.orchestra.run.smithi079.stdout:1814 trunc from 0xc9d31d9 to 0xab82e6a 2024-09-16T14:07:43.830 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:43.822+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:43.836 INFO:teuthology.orchestra.run.smithi079.stdout:1815 trunc from 0xab82e6a to 0x9eded56 2024-09-16T14:07:43.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:43.845 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:43.853 INFO:teuthology.orchestra.run.smithi079.stdout:1817 trunc from 0x9eded56 to 0x5ef6571 2024-09-16T14:07:43.864 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:43.864 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:43.870 INFO:teuthology.orchestra.run.smithi079.stdout:1818 trunc from 0x5ef6571 to 0x34b89ef 2024-09-16T14:07:43.874 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:43.895 INFO:teuthology.orchestra.run.smithi079.stdout:1819 trunc from 0x34b89ef to 0x2a25252 2024-09-16T14:07:43.920 INFO:teuthology.orchestra.run.smithi079.stdout:1820 writesame 0x631c0c7 thru 0x63229b0 data_size 0xd(0x68ea bytes) 2024-09-16T14:07:43.920 INFO:teuthology.orchestra.run.smithi079.stdout:1822 punch from 0x1ca973d to 0x1cb6c58, (0xd51b bytes) 2024-09-16T14:07:43.920 INFO:teuthology.orchestra.run.smithi079.stdout:1824 punch from 0x4d879bc to 0x4d8cacf, (0x5113 bytes) 2024-09-16T14:07:43.920 INFO:teuthology.orchestra.run.smithi079.stdout:1827 read 0x5710fe7 thru 0x5717b93 (0x6bad bytes) 2024-09-16T14:07:43.920 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:43.920 INFO:teuthology.orchestra.run.smithi079.stdout:1829 read 0x58afa79 thru 0x58b3064 (0x35ec bytes) 2024-09-16T14:07:43.920 INFO:teuthology.orchestra.run.smithi079.stdout:1831 writesame 0x4bf81d5 thru 0x4c036e1 data_size 0x1(0xb50d bytes) 2024-09-16T14:07:43.931 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:43.926+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:43.942 INFO:teuthology.orchestra.run.smithi079.stdout:1832 punch from 0xbc9559 to 0xbd25b0, (0x9057 bytes) 2024-09-16T14:07:43.942 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:43.942 INFO:teuthology.orchestra.run.smithi079.stdout:1834 punch from 0x47d8881 to 0x47e262e, (0x9dad bytes) 2024-09-16T14:07:43.942 INFO:teuthology.orchestra.run.smithi079.stdout:1835 read 0x3e5af27 thru 0x3e651f6 (0xa2d0 bytes) 2024-09-16T14:07:43.942 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:43.957 INFO:teuthology.orchestra.run.smithi079.stdout:1841 write 0xa731d2e thru 0xa732503 (0x7d6 bytes) 2024-09-16T14:07:43.963 INFO:teuthology.orchestra.run.smithi079.stdout:1842 write 0x4388b1c thru 0x4394e3c (0xc321 bytes) 2024-09-16T14:07:43.967 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:43.967 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:43.981 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:43.984 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:43.994 INFO:teuthology.orchestra.run.smithi079.stdout:1843 trunc from 0xa732504 to 0x50184e 2024-09-16T14:07:44.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:44.025 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:44.032 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:44.026+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:44.038 INFO:teuthology.orchestra.run.smithi079.stdout:1844 write 0xb946d64 thru 0xb953f8e (0xd22b bytes) 2024-09-16T14:07:44.048 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:44.050 INFO:teuthology.orchestra.run.smithi079.stdout:1845 write 0x22a06ec thru 0x22a4330 (0x3c45 bytes) 2024-09-16T14:07:44.060 INFO:teuthology.orchestra.run.smithi079.stdout:1847 trunc from 0xb953f8f to 0xe6c9eed 2024-09-16T14:07:44.068 INFO:teuthology.orchestra.run.smithi079.stdout:1849 write 0x96c8b96 thru 0x96d3641 (0xaaac bytes) 2024-09-16T14:07:44.071 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:44.080 INFO:teuthology.orchestra.run.smithi079.stdout:1852 read 0x3dd6e77 thru 0x3ddaebb (0x4045 bytes) 2024-09-16T14:07:44.080 INFO:teuthology.orchestra.run.smithi079.stdout:1853 writesame 0x3b887f thru 0x3b9b04 data_size 0x1(0x1286 bytes) 2024-09-16T14:07:44.080 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:44.080 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:44.085 INFO:teuthology.orchestra.run.smithi079.stdout:1856 write 0xddc9100 thru 0xddcfc68 (0x6b69 bytes) 2024-09-16T14:07:44.090 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:44.094 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:44.094 INFO:teuthology.orchestra.run.smithi079.stdout:1858 punch from 0x74dde25 to 0x74ec58d, (0xe768 bytes) 2024-09-16T14:07:44.094 INFO:teuthology.orchestra.run.smithi079.stdout:1859 punch from 0xaaa7790 to 0xaaabfa9, (0x4819 bytes) 2024-09-16T14:07:44.094 INFO:teuthology.orchestra.run.smithi079.stdout:1860 writesame 0x50c4565 thru 0x50c6ab8 data_size 0x2(0x2554 bytes) 2024-09-16T14:07:44.100 INFO:teuthology.orchestra.run.smithi079.stdout:1861 read 0x26db11 thru 0x270c9c (0x318c bytes) 2024-09-16T14:07:44.102 INFO:teuthology.orchestra.run.smithi079.stdout:1862 read 0x5650eb7 thru 0x565c90a (0xba54 bytes) 2024-09-16T14:07:44.102 INFO:teuthology.orchestra.run.smithi079.stdout:1864 write 0x3b4a231 thru 0x3b4d202 (0x2fd2 bytes) 2024-09-16T14:07:44.104 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:44.124 INFO:teuthology.orchestra.run.smithi079.stdout:1866 trunc from 0xe6c9eed to 0x2f3da12 2024-09-16T14:07:44.135 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:44.140 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:44.134+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:44.146 INFO:teuthology.orchestra.run.smithi079.stdout:1867 punch from 0xba1c49 to 0xbaf798, (0xdb4f bytes) 2024-09-16T14:07:44.155 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:44.157 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:44.157 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:44.157 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:44.157 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:44.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:44.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:44.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:44.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:44.160 INFO:teuthology.orchestra.run.smithi079.stdout:1869 writesame 0x6e66c08 thru 0x6e6cb76 data_size 0x1(0x5f6f bytes) 2024-09-16T14:07:44.168 INFO:teuthology.orchestra.run.smithi079.stdout:1870 trunc from 0x6e6cb77 to 0xc8d0711 2024-09-16T14:07:44.176 INFO:teuthology.orchestra.run.smithi079.stdout:1871 read 0x8aceb90 thru 0x8adbfd8 (0xd449 bytes) 2024-09-16T14:07:44.185 INFO:teuthology.orchestra.run.smithi079.stdout:1872 write 0xcf09570 thru 0xcf0c387 (0x2e18 bytes) 2024-09-16T14:07:44.186 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:44.186 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:44.186 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:44.194 INFO:teuthology.orchestra.run.smithi079.stdout:1873 read 0x8cf3b53 thru 0x8cf8eef (0x539d bytes) 2024-09-16T14:07:44.194 INFO:teuthology.orchestra.run.smithi079.stdout:1874 clone 14 order 20 su 8192 sc 15 2024-09-16T14:07:44.196 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:44.235 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:44.230+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:44.293 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9779 3.7 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.995147792000001, 2024-09-16T14:07:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042761995999999997, 2024-09-16T14:07:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9779 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293651+0000", 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.021e-06 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305709+0000", 2024-09-16T14:07:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012058627000000001 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305754+0000", 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4137000000000001e-05 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305769+0000", 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5257e-05 2024-09-16T14:07:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305971+0000", 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000201833 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.306248+0000", 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027726999999999998 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.307018+0000", 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076971799999999996 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.329087+0000", 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022069787 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336319+0000", 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072311789999999999 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336375+0000", 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6406999999999997e-05 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336410+0000", 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4759999999999999e-05 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12172 3.12 3:4dae129f:::rbd_data.11364aaed44f.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.741852504000001, 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031007818999999999, 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12172 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546946+0000", 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2270000000000002e-06 2024-09-16T14:07:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546958+0000", 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.173e-05 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546989+0000", 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1449000000000003e-05 2024-09-16T14:07:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547001+0000", 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1216999999999999e-05 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547125+0000", 2024-09-16T14:07:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012428299999999999 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547345+0000", 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022038799999999999 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.555417+0000", 2024-09-16T14:07:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080718119999999994 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577743+0000", 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02232599 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577765+0000", 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1675000000000002e-05 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577879+0000", 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011386400000000001 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577951+0000", 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2183999999999999e-05 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14144 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620414+0000", 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6683819250000003, 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022268178999999999, 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14144 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620414+0000", 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620414+0000", 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620414+0000", 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1099999999999996e-07 2024-09-16T14:07:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620416+0000", 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1229999999999998e-06 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620422+0000", 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620426+0000", 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6959999999999998e-06 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642494+0000", 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022068641999999999 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642573+0000", 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8341000000000001e-05 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642682+0000", 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108995 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14146 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620432+0000", 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6683633340000004, 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022436186, 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14146 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620432+0000", 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620432+0000", 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620433+0000", 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620436+0000", 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9909999999999998e-06 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620442+0000", 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0000000000000002e-06 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620446+0000", 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8039999999999999e-06 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642723+0000", 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022277267 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642781+0000", 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8094000000000003e-05 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642868+0000", 2024-09-16T14:07:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7507000000000001e-05 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14155 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620449+0000", 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6683461949999998, 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022645345000000001, 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14155 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620449+0000", 2024-09-16T14:07:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620449+0000", 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620450+0000", 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999999999999997e-07 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620452+0000", 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7430000000000001e-06 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620459+0000", 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3039999999999999e-06 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620463+0000", 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3719999999999998e-06 2024-09-16T14:07:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642911+0000", 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022447501000000002 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.642992+0000", 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0894000000000002e-05 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643095+0000", 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103041 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14158 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620469+0000", 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.668326253, 2024-09-16T14:07:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023038100999999998, 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14158 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620469+0000", 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620469+0000", 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620470+0000", 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.89e-07 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620472+0000", 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.906e-06 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620478+0000", 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8830000000000003e-06 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620481+0000", 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7929999999999999e-06 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643137+0000", 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022655589 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643251+0000", 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114231 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643507+0000", 2024-09-16T14:07:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025620999999999999 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14159 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 589824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620485+0000", 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6683103799999994, 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023253517000000001, 2024-09-16T14:07:44.328 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14159 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620485+0000", 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620485+0000", 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620486+0000", 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9999999999999997e-07 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620499+0000", 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2818e-05 2024-09-16T14:07:44.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620505+0000", 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2500000000000003e-06 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620509+0000", 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1110000000000001e-06 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643534+0000", 2024-09-16T14:07:44.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023024747000000002 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643620+0000", 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6522999999999998e-05 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643739+0000", 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118468 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14162 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620535+0000", 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6682609280000005, 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023321043999999999, 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14162 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620535+0000", 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620535+0000", 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620536+0000", 2024-09-16T14:07:44.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9300000000000006e-07 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620538+0000", 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1600000000000001e-06 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620545+0000", 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7850000000000003e-06 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620552+0000", 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643766+0000", 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023214067000000001 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643809+0000", 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3396999999999998e-05 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643856+0000", 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6628999999999997e-05 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14163 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620556+0000", 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6682399369999992, 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023398592999999999, 2024-09-16T14:07:44.337 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14163 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620556+0000", 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620556+0000", 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620556+0000", 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5499999999999997e-07 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620558+0000", 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T14:07:44.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620564+0000", 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9719999999999997e-06 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620568+0000", 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8890000000000004e-06 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643872+0000", 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023303707999999999 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643909+0000", 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7219999999999999e-05 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643954+0000", 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4982999999999999e-05 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14165 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620815+0000", 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6679804300000001, 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023234923000000001, 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14165 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620815+0000", 2024-09-16T14:07:44.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620815+0000", 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620816+0000", 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2529999999999999e-06 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620820+0000", 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5030000000000002e-06 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620829+0000", 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3219999999999999e-06 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620834+0000", 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8930000000000004e-06 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.643970+0000", 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023135572 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644005+0000", 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5296999999999998e-05 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644050+0000", 2024-09-16T14:07:44.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5083000000000002e-05 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14169 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620858+0000", 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6679376690000005, 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023309318999999998, 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14169 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620858+0000", 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620858+0000", 2024-09-16T14:07:44.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620858+0000", 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.13e-07 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620861+0000", 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3460000000000001e-06 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620868+0000", 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620872+0000", 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0890000000000002e-06 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644065+0000", 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023192023999999999 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644108+0000", 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3302e-05 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644167+0000", 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9398000000000003e-05 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14170 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6679193629999993, 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023454853000000001, 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14170 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620877+0000", 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1500000000000005e-07 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620879+0000", 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0269999999999998e-06 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620884+0000", 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7180000000000001e-06 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620891+0000", 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2319999999999998e-06 2024-09-16T14:07:44.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644192+0000", 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023301776 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644255+0000", 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2948999999999996e-05 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644331+0000", 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5636000000000006e-05 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14172 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6679013339999997, 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023574827999999999, 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14172 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620895+0000", 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4300000000000002e-07 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620897+0000", 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.939e-06 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620903+0000", 2024-09-16T14:07:44.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7450000000000001e-06 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620908+0000", 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7769999999999999e-06 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644351+0000", 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023442411 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644388+0000", 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7441999999999998e-05 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644469+0000", 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0771000000000003e-05 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14175 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6678807510000002, 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023718466000000001, 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14175 2024-09-16T14:07:44.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620927+0000", 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2113000000000001e-05 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620943+0000", 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:07:44.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620947+0000", 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5669999999999996e-06 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644494+0000", 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023546886999999999 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644563+0000", 2024-09-16T14:07:44.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8485000000000006e-05 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644633+0000", 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0529000000000002e-05 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14176 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:07:44.366 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6678436049999998, 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023877883999999999, 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14176 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:07:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9399999999999995e-07 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620954+0000", 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0200000000000001e-06 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620960+0000", 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8440000000000003e-06 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620964+0000", 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7459999999999998e-06 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644666+0000", 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023702375000000001 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644720+0000", 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3813000000000001e-05 2024-09-16T14:07:44.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644830+0000", 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109592 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14179 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6678241000000007, 2024-09-16T14:07:44.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024008551999999999, 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14179 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:07:44.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620972+0000", 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000003e-07 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620981+0000", 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4420000000000004e-06 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620988+0000", 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.297e-06 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620992+0000", 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8979999999999998e-06 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644865+0000", 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023872899999999999 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644910+0000", 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5794999999999998e-05 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644980+0000", 2024-09-16T14:07:44.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9708000000000004e-05 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14180 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6677999569999997, 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024190943999999999, 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14180 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:07:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0100000000000005e-07 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620998+0000", 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9829999999999999e-06 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.621004+0000", 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8270000000000003e-06 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.621009+0000", 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4530000000000001e-06 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645013+0000", 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024003295000000001 2024-09-16T14:07:44.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645077+0000", 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4139000000000004e-05 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645187+0000", 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109746 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:17276 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 3956736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:36.251735+0000", 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.0370603660000004, 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022215209999999999, 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 17276 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.381 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251735+0000", 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251735+0000", 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251736+0000", 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3200000000000004e-07 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251738+0000", 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8920000000000001e-06 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251744+0000", 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9780000000000002e-06 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251749+0000", 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4979999999999997e-06 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.273767+0000", 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022017256999999998 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.273855+0000", 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8691999999999997e-05 2024-09-16T14:07:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.273950+0000", 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5161000000000006e-05 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:17277 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 4038656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:36.251755+0000", 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.0370409939999998, 2024-09-16T14:07:44.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022353208999999999, 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 17277 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251755+0000", 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251755+0000", 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251755+0000", 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.06e-07 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251758+0000", 2024-09-16T14:07:44.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0319999999999999e-06 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251765+0000", 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7039999999999997e-06 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251768+0000", 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4240000000000002e-06 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.273971+0000", 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022202403999999999 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274030+0000", 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9750000000000002e-05 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274108+0000", 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7288999999999998e-05 2024-09-16T14:07:44.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:17278 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 3629056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:36.251773+0000", 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.037022146, 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022542092, 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 17278 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251773+0000", 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251773+0000", 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251774+0000", 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.46e-07 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251776+0000", 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.88e-06 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251781+0000", 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1039999999999998e-06 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.251785+0000", 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5159999999999996e-06 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274146+0000", 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022360604999999999 2024-09-16T14:07:44.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274226+0000", 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9837000000000001e-05 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.274316+0000", 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9703999999999994e-05 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.395 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:44.396 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:44.397 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:44.394+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:44.497 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:44.490+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:44.503 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:44.503 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:44.503 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:44.503 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:44.512 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:44.598 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:44.594+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:44.659 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:44.660 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:44.668 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:44.698 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:44.690+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.354873668, 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:07:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:07:44.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:07:44.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4066 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412417+0000", 2024-09-16T14:07:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.352393066000001, 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063826170000000002, 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4066 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412417+0000", 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412417+0000", 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412418+0000", 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7400000000000002e-07 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412421+0000", 2024-09-16T14:07:44.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0489999999999999e-06 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412429+0000", 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1620000000000008e-06 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412434+0000", 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9089999999999999e-06 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418667+0000", 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062334399999999998 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418737+0000", 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9994999999999996e-05 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418800+0000", 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2287999999999997e-05 2024-09-16T14:07:44.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4067 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 262144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412554+0000", 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.35225574, 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064644630000000002, 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.782 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4067 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412554+0000", 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412554+0000", 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412556+0000", 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.165e-06 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412561+0000", 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7400000000000004e-06 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412621+0000", 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9973000000000003e-05 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412644+0000", 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2442999999999998e-05 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418824+0000", 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061800140000000002 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.418920+0000", 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6545999999999993e-05 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419019+0000", 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8581999999999999e-05 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.786 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4068 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 344064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412655+0000", 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.352155448000001, 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065592130000000004, 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4068 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412655+0000", 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412655+0000", 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412655+0000", 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2799999999999995e-07 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412673+0000", 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.738e-05 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412697+0000", 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4634000000000002e-05 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412703+0000", 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0000000000000002e-06 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419059+0000", 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063559920000000004 2024-09-16T14:07:44.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419136+0000", 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6946000000000005e-05 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419214+0000", 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7433000000000002e-05 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4069 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 40960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412710+0000", 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.352099551999999, 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006666763, 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4069 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412710+0000", 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412710+0000", 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412711+0000", 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1100000000000005e-07 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412714+0000", 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1259999999999998e-06 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412723+0000", 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1939999999999998e-06 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412727+0000", 2024-09-16T14:07:44.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5109999999999997e-06 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419256+0000", 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065286550000000004 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419317+0000", 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1629000000000002e-05 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419377+0000", 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9837000000000003e-05 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4070 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 122880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412732+0000", 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.352077593000001, 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068543459999999999, 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4070 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412732+0000", 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412732+0000", 2024-09-16T14:07:44.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412733+0000", 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412736+0000", 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6139999999999999e-06 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412744+0000", 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8569999999999995e-06 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412748+0000", 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6349999999999997e-06 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419406+0000", 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066574879999999996 2024-09-16T14:07:44.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419490+0000", 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4263000000000003e-05 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419587+0000", 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6809000000000001e-05 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4071 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412762+0000", 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.352047901999999, 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070357520000000002, 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4071 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.801 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412762+0000", 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412762+0000", 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412763+0000", 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7100000000000001e-07 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412774+0000", 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0669000000000001e-05 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412783+0000", 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7769999999999997e-06 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412789+0000", 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2820000000000004e-06 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419614+0000", 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068253159999999997 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419733+0000", 2024-09-16T14:07:44.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119171 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419798+0000", 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4765999999999996e-05 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4072 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412794+0000", 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.352015666, 2024-09-16T14:07:44.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072234769999999998, 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4072 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412794+0000", 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412794+0000", 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412795+0000", 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5799999999999998e-07 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412797+0000", 2024-09-16T14:07:44.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2910000000000002e-06 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412805+0000", 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7260000000000002e-06 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412809+0000", 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3250000000000001e-06 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419822+0000", 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070127820000000004 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.419932+0000", 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010974199999999999 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420018+0000", 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5853000000000006e-05 2024-09-16T14:07:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4073 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 368640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.412815+0000", 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.351995272, 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074833160000000003, 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4073 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412815+0000", 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412815+0000", 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412815+0000", 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.37e-07 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412818+0000", 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3199999999999998e-06 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412825+0000", 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4800000000000004e-06 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.412830+0000", 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6e-06 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420064+0000", 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072340640000000001 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420193+0000", 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012862299999999999 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420298+0000", 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105492 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4074 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 57344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413225+0000", 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.351585081, 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073554960000000004, 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4074 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413225+0000", 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413225+0000", 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413227+0000", 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413233+0000", 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6830000000000002e-06 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413278+0000", 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4919000000000001e-05 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413297+0000", 2024-09-16T14:07:44.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8244999999999999e-05 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420357+0000", 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070600669999999997 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420472+0000", 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011545899999999999 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420580+0000", 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108263 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4075 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 139264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413305+0000", 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.351504521999999, 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075430260000000004, 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4075 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413305+0000", 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413305+0000", 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413307+0000", 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0249999999999999e-06 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413310+0000", 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.783e-06 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413319+0000", 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5799999999999992e-06 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413323+0000", 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420624+0000", 2024-09-16T14:07:44.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073002049999999997 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420716+0000", 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2466999999999994e-05 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420849+0000", 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013234 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4076 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413328+0000", 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.351481623000002, 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077674240000000002, 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.824 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4076 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413328+0000", 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413328+0000", 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413329+0000", 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.54e-07 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413332+0000", 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413341+0000", 2024-09-16T14:07:44.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5659999999999996e-06 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413346+0000", 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0649999999999998e-06 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420890+0000", 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007544083 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.420999+0000", 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010828300000000001 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421096+0000", 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7070000000000004e-05 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4077 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 303104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413352+0000", 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.351458445999999, 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079281709999999995, 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4077 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413352+0000", 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413352+0000", 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413352+0000", 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.54e-07 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413355+0000", 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3690000000000001e-06 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413364+0000", 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4110000000000002e-06 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413369+0000", 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421121+0000", 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077525240000000002 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421211+0000", 2024-09-16T14:07:44.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9825000000000001e-05 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421280+0000", 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8563000000000001e-05 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:4078 3.8 3:12b0f383:::rbd_data.1136e0d3367d.0000000000000005:5 [read 385024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:22.413374+0000", 2024-09-16T14:07:44.833 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.351435638000002, 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080617139999999993, 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4078 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413374+0000", 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413374+0000", 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413375+0000", 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413377+0000", 2024-09-16T14:07:44.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2400000000000002e-06 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413385+0000", 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8890000000000002e-06 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.413390+0000", 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6970000000000001e-06 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421310+0000", 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079205270000000001 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421388+0000", 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7106000000000003e-05 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.421436+0000", 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8544000000000001e-05 2024-09-16T14:07:44.837 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9028 3.b 3:d1d3b7e8:::rbd_data.1136dd12cea0.0000000000000023:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:25.817264+0000", 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.947545557000002, 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010794115999999999, 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9028 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817264+0000", 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817264+0000", 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817267+0000", 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817281+0000", 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4104000000000001e-05 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817329+0000", 2024-09-16T14:07:44.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7669e-05 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817350+0000", 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0735000000000001e-05 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817519+0000", 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016912700000000001 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:07:44.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.817529+0000", 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0266e-05 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.823533+0000", 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060039530000000002 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.823660+0000", 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000126365 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.823987+0000", 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032733600000000001 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.827260+0000", 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032734090000000001 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.827984+0000", 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00072412099999999999 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.828016+0000", 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1809e-05 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.828059+0000", 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2286000000000001e-05 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.844 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9033 3.b 3:d480079a:::rbd_data.1136dd12cea0.0000000000000032:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:25.822388+0000", 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.942421547999999, 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069682609999999999, 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9033 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822388+0000", 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822388+0000", 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822391+0000", 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6299999999999998e-06 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822404+0000", 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3135999999999999e-05 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822449+0000", 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4489e-05 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822468+0000", 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8899e-05 2024-09-16T14:07:44.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822684+0000", 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000216293 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.822696+0000", 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1610000000000001e-05 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.825986+0000", 2024-09-16T14:07:44.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032903450000000001 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.826092+0000", 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106159 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.826386+0000", 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029419899999999998 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.827595+0000", 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001208403 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.829256+0000", 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016611169999999999 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.829307+0000", 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0803000000000001e-05 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:25.829357+0000", 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0178e-05 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:13220 3.8 3:17c989a2:::rbd_data.11361fa8f1cb.0000000000000170:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:34.811594+0000", 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.9532158919999993, 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077263310000000003, 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13220 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.811594+0000", 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.811594+0000", 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.811596+0000", 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6980000000000001e-06 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.811605+0000", 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8820000000000007e-06 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.811625+0000", 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0375999999999999e-05 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.811639+0000", 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4262e-05 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.811789+0000", 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015010300000000001 2024-09-16T14:07:44.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.812080+0000", 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029070000000000002 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.812656+0000", 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00057635200000000003 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.815906+0000", 2024-09-16T14:07:44.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032493890000000001 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.819069+0000", 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031630249999999999 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.819248+0000", 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017866299999999999 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:34.819320+0000", 2024-09-16T14:07:44.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880999999999996e-05 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:19174 3.b 3:d6f80c72:::rbd_data.11362defb50e.0000000000000133:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:36.633127+0000", 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.1316832120000004, 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071968099999999997, 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19174 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633127+0000", 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633127+0000", 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633129+0000", 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3e-06 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633140+0000", 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1257000000000001e-05 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633181+0000", 2024-09-16T14:07:44.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0160999999999997e-05 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633199+0000", 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8369999999999999e-05 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633272+0000", 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2797000000000001e-05 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633349+0000", 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6911999999999998e-05 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.633453+0000", 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010491400000000001 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.635250+0000", 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001796925 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.640109+0000", 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004858696 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.640232+0000", 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000122881 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.640324+0000", 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1596999999999997e-05 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.862 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:19181 3.b 3:d08c0d7a:::rbd_data.11362defb50e.000000000000014b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:36.636841+0000", 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.1279691570000008, 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067588759999999996, 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 19181 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.636841+0000", 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.636841+0000", 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.636843+0000", 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.636852+0000", 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7329999999999998e-06 2024-09-16T14:07:44.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.636873+0000", 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0290000000000001e-05 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.636885+0000", 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.275e-05 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.637404+0000", 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051875100000000002 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.637608+0000", 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020331799999999999 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.637969+0000", 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036161899999999998 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.641188+0000", 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003219277 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.643516+0000", 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023280610000000002 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.643565+0000", 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8733999999999998e-05 2024-09-16T14:07:44.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:36.643600+0000", 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4483000000000002e-05 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27482 3.b 3:d59fc11a:::rbd_id.image_client.0-clone12:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:40.828114+0000", 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.9366962569999999, 2024-09-16T14:07:44.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072589669999999999, 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27482 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:44.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.828114+0000", 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.828114+0000", 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:44.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.828116+0000", 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4119999999999999e-06 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.828126+0000", 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0200999999999999e-05 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.828156+0000", 2024-09-16T14:07:44.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0097999999999999e-05 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.828167+0000", 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0149999999999999e-05 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.828238+0000", 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1694999999999997e-05 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:07:44.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.828245+0000", 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1890000000000001e-06 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.831940+0000", 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036943409999999999 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.832128+0000", 2024-09-16T14:07:44.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000187734 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.832345+0000", 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021706699999999999 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.833471+0000", 2024-09-16T14:07:44.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011266049999999999 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.835278+0000", 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018066099999999999 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.835330+0000", 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2095000000000002e-05 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:44.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:44.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.835373+0000", 2024-09-16T14:07:44.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2769999999999999e-05 2024-09-16T14:07:44.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.877 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:44.877 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:44.877 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:44.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:07:44.879 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T14:07:44.880 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:44.874+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:44.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T14:07:44.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:44.915 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone13 from 0xcf0c388 (overlap 0x33e9c6) to 0x19f4e3 2024-09-16T14:07:44.980 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:44.974+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:45.047 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-16T14:07:45.055 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:45.055 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:45.056 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:45.056 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:45.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:45.081 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:45.074+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:45.164 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:45.164 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:45.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:45.218 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:45.214+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:45.282 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:45.274+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:45.339 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5085 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 16384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.620791696000001, 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015613336, 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5085 2024-09-16T14:07:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715330+0000", 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0200000000000003e-07 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715333+0000", 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715339+0000", 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3629999999999999e-06 2024-09-16T14:07:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715343+0000", 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.924e-06 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.729409+0000", 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014065846999999999 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.730871+0000", 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014620119999999999 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.730943+0000", 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2521000000000006e-05 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5086 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 98304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.620774916999999, 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015853791999999998, 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5086 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715347+0000", 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3799999999999997e-07 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715349+0000", 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.136e-06 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715356+0000", 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1290000000000001e-06 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715359+0000", 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.495e-06 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731021+0000", 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015662367999999999 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731115+0000", 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3377999999999999e-05 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731201+0000", 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5748000000000006e-05 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5087 3.3 3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.620758520999999, 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016037577000000001, 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5087 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715363+0000", 2024-09-16T14:07:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715364+0000", 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5100000000000001e-07 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715366+0000", 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715371+0000", 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3469999999999998e-06 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715374+0000", 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4450000000000001e-06 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731249+0000", 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015874528999999998 2024-09-16T14:07:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731332+0000", 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731401+0000", 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9209999999999996e-05 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5097 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.620693267, 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016148116000000001, 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5097 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715428+0000", 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715429+0000", 2024-09-16T14:07:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0500000000000004e-07 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715432+0000", 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9569999999999998e-06 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715438+0000", 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2519999999999996e-06 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715442+0000", 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8130000000000002e-06 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731427+0000", 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015984851000000001 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731498+0000", 2024-09-16T14:07:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1260999999999994e-05 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731577+0000", 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8477000000000001e-05 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:5098 3.3 3:c0977406:::rbd_data.1136494ba971.000000000000000f:7 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.620675822999999, 2024-09-16T14:07:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016302106, 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 5098 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715446+0000", 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715448+0000", 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.852e-06 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715455+0000", 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3720000000000001e-06 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.715459+0000", 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0990000000000001e-06 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731602+0000", 2024-09-16T14:07:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016142897 2024-09-16T14:07:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731668+0000", 2024-09-16T14:07:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6161999999999994e-05 2024-09-16T14:07:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.731748+0000", 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0314000000000004e-05 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14147 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 491520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621244+0000", 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7148779520000001, 2024-09-16T14:07:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015421412000000001, 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14147 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621244+0000", 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621244+0000", 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621244+0000", 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3900000000000005e-07 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621248+0000", 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1879999999999998e-06 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621255+0000", 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7800000000000001e-06 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621265+0000", 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3340000000000008e-06 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636574+0000", 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015309852000000001 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636614+0000", 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9614999999999999e-05 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636665+0000", 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1103999999999997e-05 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14148 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 983040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621270+0000", 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7148515409999998, 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015529344, 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14148 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621270+0000", 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621270+0000", 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621271+0000", 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4700000000000001e-07 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621274+0000", 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.892e-06 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621284+0000", 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0796e-05 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621289+0000", 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7180000000000004e-06 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636688+0000", 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015398474000000001 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636730+0000", 2024-09-16T14:07:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2330000000000003e-05 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636800+0000", 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9586999999999996e-05 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14156 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621309+0000", 2024-09-16T14:07:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7148127750000004, 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015606538999999999, 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14156 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621309+0000", 2024-09-16T14:07:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621309+0000", 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621310+0000", 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3900000000000004e-07 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621313+0000", 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3730000000000001e-06 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621325+0000", 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2092e-05 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621330+0000", 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2700000000000004e-06 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636826+0000", 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015496025 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636864+0000", 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7092000000000003e-05 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636916+0000", 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1947999999999998e-05 2024-09-16T14:07:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14157 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 524288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621336+0000", 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7147858899999999, 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015693817999999998, 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14157 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621336+0000", 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621336+0000", 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621336+0000", 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8500000000000001e-07 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621339+0000", 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:07:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621350+0000", 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.064e-05 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621354+0000", 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5129999999999998e-06 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636939+0000", 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015584275 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.636978+0000", 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9576999999999997e-05 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637030+0000", 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.151e-05 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14160 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621360+0000", 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7147618970000007, 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015790060000000002, 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14160 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621360+0000", 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621360+0000", 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621360+0000", 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4300000000000003e-07 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621363+0000", 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.835e-06 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621374+0000", 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0874e-05 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621379+0000", 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6269999999999999e-06 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637065+0000", 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015686169999999999 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637101+0000", 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5973000000000003e-05 2024-09-16T14:07:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637150+0000", 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9038e-05 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14161 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 606208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621384+0000", 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7147377489999993, 2024-09-16T14:07:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015880874, 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14161 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621384+0000", 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621384+0000", 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621385+0000", 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1799999999999995e-07 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621388+0000", 2024-09-16T14:07:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0350000000000002e-06 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621398+0000", 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0560999999999999e-05 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621403+0000", 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5600000000000004e-06 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637167+0000", 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015763876999999999 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637206+0000", 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9101999999999999e-05 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637265+0000", 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9120999999999999e-05 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14164 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621409+0000", 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7147130879999999, 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015956782999999999, 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14164 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621409+0000", 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621409+0000", 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621409+0000", 2024-09-16T14:07:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621412+0000", 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5459999999999998e-06 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621422+0000", 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0454e-05 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621427+0000", 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7310000000000001e-06 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637284+0000", 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0158569 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637322+0000", 2024-09-16T14:07:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8056999999999998e-05 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637365+0000", 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3569999999999998e-05 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14166 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621432+0000", 2024-09-16T14:07:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7146897600000006, 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016113662000000001, 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14166 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621432+0000", 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621432+0000", 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621432+0000", 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9500000000000003e-07 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621435+0000", 2024-09-16T14:07:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5830000000000001e-06 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621446+0000", 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0916e-05 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621450+0000", 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4449999999999998e-06 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637388+0000", 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015937409999999999 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637446+0000", 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7917999999999997e-05 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637546+0000", 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9895000000000005e-05 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14168 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 770048~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621456+0000", 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7146653619999999, 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016249660999999999, 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14168 2024-09-16T14:07:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621456+0000", 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621456+0000", 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621457+0000", 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6100000000000001e-07 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621459+0000", 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5299999999999999e-06 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621470+0000", 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0823e-05 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621484+0000", 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3920999999999999e-05 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637571+0000", 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016087108999999999 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637626+0000", 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4576000000000001e-05 2024-09-16T14:07:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637706+0000", 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0241000000000006e-05 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14171 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621509+0000", 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7146131310000001, 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016422005, 2024-09-16T14:07:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14171 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621509+0000", 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621509+0000", 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621509+0000", 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6499999999999998e-07 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621513+0000", 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6179999999999999e-06 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621535+0000", 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2056999999999999e-05 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621539+0000", 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2799999999999997e-06 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637733+0000", 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01619365 2024-09-16T14:07:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637834+0000", 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101456 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637931+0000", 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6179000000000002e-05 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14173 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621544+0000", 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7145773890000005, 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016536887, 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14173 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621544+0000", 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621544+0000", 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621545+0000", 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:07:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621547+0000", 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2809999999999998e-06 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621553+0000", 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3419999999999996e-06 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621557+0000", 2024-09-16T14:07:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.534e-06 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.637955+0000", 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016397998 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638022+0000", 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6650999999999996e-05 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638081+0000", 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9639000000000003e-05 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14174 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 851968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621561+0000", 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7145610520000005, 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016737696999999999, 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14174 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621561+0000", 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621561+0000", 2024-09-16T14:07:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621561+0000", 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0999999999999999e-07 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621563+0000", 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9350000000000001e-06 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621572+0000", 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4200000000000007e-06 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621575+0000", 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6669999999999998e-06 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638143+0000", 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016567578999999999 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638211+0000", 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8150000000000003e-05 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638298+0000", 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7435999999999994e-05 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14177 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621579+0000", 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7145424560000002, 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016879050999999999, 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14177 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621579+0000", 2024-09-16T14:07:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621579+0000", 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621580+0000", 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621582+0000", 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.384e-06 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621591+0000", 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0100000000000001e-06 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621595+0000", 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8399999999999997e-06 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638313+0000", 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016718355000000001 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638384+0000", 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0899e-05 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638458+0000", 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4164000000000003e-05 2024-09-16T14:07:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14178 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621599+0000", 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.7145226309999995, 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017007395000000002, 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14178 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621599+0000", 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621599+0000", 2024-09-16T14:07:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621600+0000", 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621602+0000", 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1720000000000001e-06 2024-09-16T14:07:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621611+0000", 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8230000000000001e-06 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621614+0000", 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.873e-06 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638508+0000", 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016893923000000002 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638553+0000", 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4416999999999998e-05 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638607+0000", 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3733000000000002e-05 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14181 3.f 3:f2d8c91b:::rbd_data.11364aaed44f.0000000000000001:c [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.621619+0000", 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.714503101, 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017148988, 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14181 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621619+0000", 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621619+0000", 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621619+0000", 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.75e-07 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621622+0000", 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4389999999999999e-06 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621631+0000", 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9220000000000003e-06 2024-09-16T14:07:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.621634+0000", 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.704e-06 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638634+0000", 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016999538000000002 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638681+0000", 2024-09-16T14:07:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7182000000000003e-05 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.638768+0000", 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6627999999999998e-05 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:45.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:45.434 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:45.430+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:45.491 INFO:teuthology.orchestra.run.smithi079.stdout:1875 write 0x2835ce9 thru 0x283e362 (0x867a bytes) 2024-09-16T14:07:45.518 INFO:teuthology.orchestra.run.smithi079.stdout:1876 read 0x102565 thru 0x106803 (0x429f bytes) 2024-09-16T14:07:45.523 INFO:teuthology.orchestra.run.smithi079.stdout:1877 punch from 0x6190202 to 0x619deb6, (0xdcb4 bytes) 2024-09-16T14:07:45.532 INFO:teuthology.orchestra.run.smithi079.stdout:1879 trunc from 0xcf0c388 to 0x630eeb1 2024-09-16T14:07:45.536 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:45.530+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:45.549 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:45.549 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:45.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:45.570 INFO:teuthology.orchestra.run.smithi079.stdout:1881 write 0x15a6394 thru 0x15b33dd (0xd04a bytes) 2024-09-16T14:07:45.587 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:45.588 INFO:teuthology.orchestra.run.smithi079.stdout:1883 read 0x4af1c71 thru 0x4af2e22 (0x11b2 bytes) 2024-09-16T14:07:45.588 INFO:teuthology.orchestra.run.smithi079.stdout:1884 writesame 0x1559ebe thru 0x155a1a6 data_size 0x1(0x2e9 bytes) 2024-09-16T14:07:45.617 INFO:teuthology.orchestra.run.smithi079.stdout:1885 writesame 0xab1fe59 thru 0xab21564 data_size 0x14(0x170c bytes) 2024-09-16T14:07:45.617 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:45.630 INFO:teuthology.orchestra.run.smithi079.stdout:1887 writesame 0xbda4c2f thru 0xbda7734 data_size 0x1(0x2b06 bytes) 2024-09-16T14:07:45.630 INFO:teuthology.orchestra.run.smithi079.stdout:1888 writesame 0x11175da thru 0x1125483 data_size 0xb(0xdeaa bytes) 2024-09-16T14:07:45.636 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:45.630+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:45.651 INFO:teuthology.orchestra.run.smithi079.stdout:1889 trunc from 0xbda7735 to 0x6065ec0 2024-09-16T14:07:45.666 INFO:teuthology.orchestra.run.smithi079.stdout:1891 punch from 0x448cca4 to 0x449c67b, (0xf9d7 bytes) 2024-09-16T14:07:45.666 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:45.666 INFO:teuthology.orchestra.run.smithi079.stdout:1893 read 0x57ff51c thru 0x5807954 (0x8439 bytes) 2024-09-16T14:07:45.672 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:45.672 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:45.676 INFO:teuthology.orchestra.run.smithi079.stdout:1894 trunc from 0x6065ec0 to 0xb738c9e 2024-09-16T14:07:45.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:45.688 INFO:teuthology.orchestra.run.smithi079.stdout:1896 read 0x390bbc2 thru 0x3914d83 (0x91c2 bytes) 2024-09-16T14:07:45.690 INFO:teuthology.orchestra.run.smithi079.stdout:1898 punch from 0xa29b1cc to 0xa2a9909, (0xe73d bytes) 2024-09-16T14:07:45.690 INFO:teuthology.orchestra.run.smithi079.stdout:1899 read 0x585877b thru 0x5859304 (0xb8a bytes) 2024-09-16T14:07:45.691 INFO:teuthology.orchestra.run.smithi079.stdout:1900 read 0x43b67c3 thru 0x43ba72b (0x3f69 bytes) 2024-09-16T14:07:45.692 INFO:teuthology.orchestra.run.smithi079.stdout:1902 trunc from 0xb738c9e to 0xb9a9f1a 2024-09-16T14:07:45.715 INFO:teuthology.orchestra.run.smithi079.stdout:1903 writesame 0x15625ea thru 0x1568bed data_size 0x2(0x6604 bytes) 2024-09-16T14:07:45.730 INFO:teuthology.orchestra.run.smithi079.stdout:1904 trunc from 0xb9a9f1a to 0xcbcce9e 2024-09-16T14:07:45.736 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:45.730+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:45.741 INFO:teuthology.orchestra.run.smithi079.stdout:1905 read 0x53a3856 thru 0x53afc61 (0xc40c bytes) 2024-09-16T14:07:45.743 INFO:teuthology.orchestra.run.smithi079.stdout:1906 write 0x7ebcb02 thru 0x7ec9c36 (0xd135 bytes) 2024-09-16T14:07:45.767 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:45.767 INFO:teuthology.orchestra.run.smithi079.stdout:1910 write 0x93efb26 thru 0x93f3f6a (0x4445 bytes) 2024-09-16T14:07:45.777 INFO:teuthology.orchestra.run.smithi079.stdout:1912 read 0x9cdba97 thru 0x9ceafb1 (0xf51b bytes) 2024-09-16T14:07:45.790 INFO:teuthology.orchestra.run.smithi079.stdout:1913 write 0xdfa9a40 thru 0xdfacf3f (0x3500 bytes) 2024-09-16T14:07:45.798 INFO:teuthology.orchestra.run.smithi079.stdout:1914 write 0x7bc5414 thru 0x7bd17d8 (0xc3c5 bytes) 2024-09-16T14:07:45.802 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:45.802 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:45.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.949832319999999, 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15070547300000001, 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847226+0000", 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.204e-06 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847235+0000", 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847250+0000", 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5662e-05 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847261+0000", 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0236e-05 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847327+0000", 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.622e-05 2024-09-16T14:07:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847345+0000", 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8434e-05 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.993706+0000", 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14636041899999999 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994117+0000", 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041132499999999998 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994428+0000", 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031060899999999999 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.995845+0000", 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014173359999999999 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997768+0000", 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001922991 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997867+0000", 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9057999999999999e-05 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997929+0000", 2024-09-16T14:07:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2433000000000003e-05 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.570474697999998, 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015154025999999999, 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 50 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226710+0000", 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012769999999999999 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226721+0000", 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226726+0000", 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6459999999999998e-06 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238128+0000", 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011401390000000001 2024-09-16T14:07:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238249+0000", 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121284 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238407+0000", 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015738899999999999 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240509+0000", 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002102923 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241459+0000", 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094996999999999996 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241621+0000", 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000161502 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241736+0000", 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011458799999999999 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.570321773000003, 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015305664, 2024-09-16T14:07:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 51 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226735+0000", 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5700000000000001e-07 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226740+0000", 2024-09-16T14:07:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6460000000000001e-06 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226748+0000", 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226753+0000", 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6229999999999996e-06 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239228+0000", 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012475119999999999 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239405+0000", 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177641 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239559+0000", 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015368599999999999 2024-09-16T14:07:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241393+0000", 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018340260000000001 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241863+0000", 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047025300000000001 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241934+0000", 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0229999999999997e-05 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242040+0000", 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106568 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.570295379000001, 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015737872, 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 52 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226762+0000", 2024-09-16T14:07:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.23e-07 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226766+0000", 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.668e-06 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226774+0000", 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8939999999999993e-06 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226779+0000", 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1170000000000003e-06 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240584+0000", 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013804344 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240709+0000", 2024-09-16T14:07:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012524300000000001 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241014+0000", 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030534399999999998 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242062+0000", 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010479899999999999 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242369+0000", 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030710399999999999 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242442+0000", 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2383000000000001e-05 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242499+0000", 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:07:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.025980394999998, 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015694993000000001, 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 72 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771077+0000", 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0299999999999997e-07 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771209+0000", 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132347 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771224+0000", 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.454e-05 2024-09-16T14:07:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771230+0000", 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782272+0000", 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011041455 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782402+0000", 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013039099999999999 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782645+0000", 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024303500000000001 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786509+0000", 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003863698 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786628+0000", 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011894499999999999 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786711+0000", 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2961999999999996e-05 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786771+0000", 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0098e-05 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.025817666000002, 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016692631999999999, 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 73 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771240+0000", 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.569999999999999e-07 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771244+0000", 2024-09-16T14:07:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771253+0000", 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8629999999999997e-06 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771262+0000", 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783924+0000", 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012662479000000001 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784015+0000", 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0624999999999994e-05 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784129+0000", 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113563 2024-09-16T14:07:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787598+0000", 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034694890000000001 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787707+0000", 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109341 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787840+0000", 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132421 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787931+0000", 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1291000000000004e-05 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.025787569999999, 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017320314999999999, 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771278+0000", 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3790000000000006e-06 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771286+0000", 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5199999999999997e-06 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771292+0000", 2024-09-16T14:07:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2839999999999997e-06 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784933+0000", 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013641304 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785019+0000", 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6201999999999999e-05 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785164+0000", 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014457599999999999 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787609+0000", 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024456769999999998 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788403+0000", 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079383000000000003 2024-09-16T14:07:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788490+0000", 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7231000000000006e-05 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788589+0000", 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8608000000000002e-05 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.025757859999999, 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017489688, 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75 2024-09-16T14:07:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771299+0000", 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3e-07 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771421+0000", 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121544 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771433+0000", 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853e-05 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771443+0000", 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0229999999999998e-06 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785601+0000", 2024-09-16T14:07:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014158145 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785677+0000", 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6711999999999993e-05 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785774+0000", 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6583999999999995e-05 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787633+0000", 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001859461 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788688+0000", 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010546349999999999 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788747+0000", 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.923e-05 2024-09-16T14:07:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788788+0000", 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0771e-05 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.025601279999997, 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017898397999999999, 2024-09-16T14:07:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 76 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771456+0000", 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7900000000000007e-07 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771466+0000", 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8800000000000003e-06 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771475+0000", 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1670000000000005e-06 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771480+0000", 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.49e-06 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786862+0000", 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015381807000000001 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786985+0000", 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012302099999999999 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787142+0000", 2024-09-16T14:07:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015696500000000001 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788157+0000", 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001014923 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789175+0000", 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010175200000000001 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789259+0000", 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4578000000000002e-05 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789353+0000", 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4067999999999995e-05 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:7540 3.10 3:0c0a7e4b:::rbd_data.11369af4c40c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.990335525999999, 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016499916, 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7540 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806722+0000", 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.78e-07 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807847+0000", 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001125381 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807858+0000", 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1219999999999999e-05 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807863+0000", 2024-09-16T14:07:45.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1569999999999999e-06 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808087+0000", 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022387000000000001 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808219+0000", 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013168199999999999 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.809044+0000", 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082524200000000001 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.814222+0000", 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005177414 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823075+0000", 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088537470000000004 2024-09-16T14:07:45.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823150+0000", 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4571000000000001e-05 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823221+0000", 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0654000000000005e-05 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9727 3.1a 3:5b31ba1a:::rbd_data.1136a719415e.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.585745702000001, 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.033339056999999998, 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9727 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211313+0000", 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1969999999999999e-06 2024-09-16T14:07:45.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216419+0000", 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051063580000000001 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216461+0000", 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2228000000000003e-05 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216473+0000", 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1635e-05 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216560+0000", 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6745999999999999e-05 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216730+0000", 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017044299999999999 2024-09-16T14:07:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.217504+0000", 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077432 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.232742+0000", 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015237611 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244490+0000", 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011747506 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244586+0000", 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6106000000000004e-05 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244650+0000", 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3906999999999996e-05 2024-09-16T14:07:45.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9793 3.1a 3:5f64584f:::rbd_data.1136a719415e.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.448657280999999, 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024339903, 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9793 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348404+0000", 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.288e-06 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352536+0000", 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041315750000000002 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352576+0000", 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0309999999999999e-05 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352590+0000", 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3524e-05 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352690+0000", 2024-09-16T14:07:45.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010055700000000001 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352873+0000", 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000182714 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.353759+0000", 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088597700000000003 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.362898+0000", 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091386360000000003 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372631+0000", 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097337159999999999 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372700+0000", 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9016999999999993e-05 2024-09-16T14:07:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372739+0000", 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8588999999999999e-05 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14483 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701324+0000", 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.095732493, 2024-09-16T14:07:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014535807, 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14483 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701324+0000", 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701324+0000", 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701324+0000", 2024-09-16T14:07:45.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701327+0000", 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.055e-06 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701336+0000", 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3620000000000006e-06 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701340+0000", 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5410000000000002e-06 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.706650+0000", 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053099189999999998 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.706713+0000", 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3131000000000003e-05 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.707047+0000", 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033344199999999998 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.711603+0000", 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045564689999999996 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.715763+0000", 2024-09-16T14:07:45.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041601169999999996 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.715823+0000", 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9601000000000001e-05 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.715860+0000", 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6683000000000001e-05 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14484 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701345+0000", 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.095710897, 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014697109, 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14484 2024-09-16T14:07:45.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701345+0000", 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701345+0000", 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701346+0000", 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701349+0000", 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0290000000000001e-06 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701356+0000", 2024-09-16T14:07:45.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4580000000000004e-06 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701361+0000", 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5240000000000002e-06 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.707423+0000", 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060624650000000004 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.707496+0000", 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3059000000000006e-05 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.707777+0000", 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028042799999999999 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.712956+0000", 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051787339999999999 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.715894+0000", 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00293882 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716009+0000", 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114549 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716042+0000", 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3513000000000003e-05 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14485 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701366+0000", 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.095690303, 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01479137, 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14485 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701366+0000", 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701366+0000", 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701366+0000", 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8599999999999998e-07 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701369+0000", 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7939999999999998e-06 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701376+0000", 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2710000000000003e-06 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701381+0000", 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7140000000000001e-06 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708267+0000", 2024-09-16T14:07:45.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068860049999999997 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708334+0000", 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7218000000000006e-05 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708552+0000", 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021770700000000001 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.714111+0000", 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055590980000000002 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716080+0000", 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019685729999999999 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716122+0000", 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2540999999999998e-05 2024-09-16T14:07:45.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716157+0000", 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4962999999999997e-05 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14486 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701386+0000", 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.095669808, 2024-09-16T14:07:45.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014881439, 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14486 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701386+0000", 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701386+0000", 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701387+0000", 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9599999999999999e-07 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701390+0000", 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509999999999999e-06 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701397+0000", 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6440000000000003e-06 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701402+0000", 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7129999999999996e-06 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708851+0000", 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0074494069999999999 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708927+0000", 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5766000000000006e-05 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.709252+0000", 2024-09-16T14:07:45.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032530599999999998 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.714139+0000", 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048873240000000002 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716192+0000", 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020529609999999998 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716234+0000", 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1823000000000003e-05 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716268+0000", 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3748000000000003e-05 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.904 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14487 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701407+0000", 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.095649542, 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014969764, 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14487 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701407+0000", 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701407+0000", 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701407+0000", 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8800000000000002e-07 2024-09-16T14:07:45.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701434+0000", 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6961000000000002e-05 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701445+0000", 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.08e-05 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701451+0000", 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6899999999999997e-06 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.710059+0000", 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086083640000000003 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.710327+0000", 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026832899999999999 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.711146+0000", 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081821400000000005 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.714148+0000", 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030022680000000002 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716301+0000", 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021534919999999999 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716344+0000", 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2395999999999999e-05 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716376+0000", 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2662e-05 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.910 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14762 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.772489+0000", 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.024567230000001, 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015045655, 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14762 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772489+0000", 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772489+0000", 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772490+0000", 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.049e-06 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772497+0000", 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9550000000000003e-06 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772510+0000", 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3351e-05 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772516+0000", 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1179999999999997e-06 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.779908+0000", 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073911519999999998 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.780005+0000", 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7806999999999994e-05 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.780422+0000", 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041635199999999999 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.784878+0000", 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044558790000000003 2024-09-16T14:07:45.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787434+0000", 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025559060000000002 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787498+0000", 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3997000000000004e-05 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787535+0000", 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7088999999999997e-05 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14763 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.772523+0000", 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.024533030000001, 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015125418, 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14763 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772523+0000", 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772523+0000", 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772524+0000", 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1999999999999998e-07 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772528+0000", 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.301e-06 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772545+0000", 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7178999999999999e-05 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772551+0000", 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7610000000000004e-06 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.780943+0000", 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083913719999999994 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.781033+0000", 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0451999999999994e-05 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.781446+0000", 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000412493 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.784885+0000", 2024-09-16T14:07:45.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003439058 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787594+0000", 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027093709999999999 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787623+0000", 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9502e-05 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787649+0000", 2024-09-16T14:07:45.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5109e-05 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:19008 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.395517+0000", 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.4015388400000006, 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015791547, 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:45.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 19008 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395517+0000", 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:45.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395517+0000", 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395518+0000", 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9700000000000002e-07 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395520+0000", 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.531e-06 2024-09-16T14:07:45.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395526+0000", 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5489999999999997e-06 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395532+0000", 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6810000000000003e-06 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:45.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.405294+0000", 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097622330000000004 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.405426+0000", 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013181099999999999 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.405952+0000", 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052624799999999999 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.408258+0000", 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023066480000000001 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.411197+0000", 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029387110000000001 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.411247+0000", 2024-09-16T14:07:45.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9938999999999997e-05 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.411309+0000", 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1798999999999996e-05 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:45.929 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:45.930 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T14:07:45.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:45.932 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:07:45.932 INFO:teuthology.orchestra.run.smithi079.stdout:1915 punch from 0xb8dd951 to 0xb8e5aac, (0x815b bytes) 2024-09-16T14:07:45.932 INFO:teuthology.orchestra.run.smithi079.stdout:1916 read 0xd1fafa3 thru 0xd2086d3 (0xd731 bytes) 2024-09-16T14:07:45.932 INFO:teuthology.orchestra.run.smithi079.stdout:1917 read 0xd0cd28e thru 0xd0dce38 (0xfbab bytes) 2024-09-16T14:07:45.933 INFO:teuthology.orchestra.run.smithi079.stdout:1918 write 0x458b4d8 thru 0x45976b9 (0xc1e2 bytes) 2024-09-16T14:07:45.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T14:07:45.935 INFO:teuthology.orchestra.run.smithi079.stdout:1919 write 0x32f329d thru 0x32f8b6f (0x58d3 bytes) 2024-09-16T14:07:45.935 INFO:teuthology.orchestra.run.smithi079.stdout:1921 writesame 0x3968fb9 thru 0x3971b48 data_size 0x70(0x8b90 bytes) 2024-09-16T14:07:45.935 INFO:teuthology.orchestra.run.smithi079.stdout:1923 trunc from 0xdfacf40 to 0x66b6b76 2024-09-16T14:07:45.935 INFO:teuthology.orchestra.run.smithi079.stdout:1924 writesame 0xa2a561 thru 0xa2f3be data_size 0x7(0x4e5e bytes) 2024-09-16T14:07:45.935 INFO:teuthology.orchestra.run.smithi079.stdout:1925 write 0x89eb5d3 thru 0x89f7933 (0xc361 bytes) 2024-09-16T14:07:45.935 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:45.935 INFO:teuthology.orchestra.run.smithi079.stdout:1927 read 0x4c058fe thru 0x4c135a8 (0xdcab bytes) 2024-09-16T14:07:45.935 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:45.936 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:45.922+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:45.951 INFO:teuthology.orchestra.run.smithi079.stdout:1929 writesame 0xc96eef1 thru 0xc979977 data_size 0x1(0xaa87 bytes) 2024-09-16T14:07:45.952 INFO:teuthology.orchestra.run.smithi079.stdout:1930 write 0x240b756 thru 0x240ba44 (0x2ef bytes) 2024-09-16T14:07:45.961 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:45.961 INFO:teuthology.orchestra.run.smithi079.stdout:1932 write 0x32acbb6 thru 0x32bc0f1 (0xf53c bytes) 2024-09-16T14:07:45.976 INFO:teuthology.orchestra.run.smithi079.stdout:1934 write 0x300aab3 thru 0x3018fe5 (0xe533 bytes) 2024-09-16T14:07:45.986 INFO:teuthology.orchestra.run.smithi079.stdout:1935 writesame 0x32adb95 thru 0x32adc52 data_size 0x5(0xbe bytes) 2024-09-16T14:07:45.988 INFO:teuthology.orchestra.run.smithi079.stdout:1936 read 0xadbf011 thru 0xadcb337 (0xc327 bytes) 2024-09-16T14:07:45.988 INFO:teuthology.orchestra.run.smithi079.stdout:1937 read 0x90d85a6 thru 0x90e5053 (0xcaae bytes) 2024-09-16T14:07:45.989 INFO:teuthology.orchestra.run.smithi079.stdout:1938 punch from 0x7a12663 to 0x7a174e5, (0x4e82 bytes) 2024-09-16T14:07:45.989 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:45.989 INFO:teuthology.orchestra.run.smithi079.stdout:1941 read 0x169b1d1 thru 0x169b2ee (0x11e bytes) 2024-09-16T14:07:45.998 INFO:teuthology.orchestra.run.smithi079.stdout:1942 trunc from 0xc979978 to 0x818ed87 2024-09-16T14:07:46.019 INFO:teuthology.orchestra.run.smithi079.stdout:1944 writesame 0x6ad68be thru 0x6ae6262 data_size 0x51(0xf9a5 bytes) 2024-09-16T14:07:46.031 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:46.026+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:46.042 INFO:teuthology.orchestra.run.smithi079.stdout:1946 punch from 0x7e83bbd to 0x7e84eaa, (0x12ed bytes) 2024-09-16T14:07:46.042 INFO:teuthology.orchestra.run.smithi079.stdout:1947 read 0x5f6c784 thru 0x5f6d07a (0x8f7 bytes) 2024-09-16T14:07:46.042 INFO:teuthology.orchestra.run.smithi079.stdout:1950 writesame 0x271573 thru 0x280760 data_size 0x2(0xf1ee bytes) 2024-09-16T14:07:46.067 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:30520 3.11 e52/23 3:8dbd2ffb:::rbd_data.11369f5c332a.0000000000000001:head v 52'150, mlcod=52'148)", 2024-09-16T14:07:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:46.066012+0000", 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00040173599999999998, 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00042228099999999999, 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:46.066012+0000", 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:46.066012+0000", 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:46.066013+0000", 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3039999999999999e-06 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:46.066041+0000", 2024-09-16T14:07:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8245000000000001e-05 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:46.066064+0000", 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2654e-05 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:46.066073+0000", 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7190000000000007e-06 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:46.066119+0000", 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6390000000000001e-05 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:46.066141+0000", 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2347000000000001e-05 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-16T14:07:46.071 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:46.074 INFO:teuthology.orchestra.run.smithi079.stdout:1951 write 0x6f00485 thru 0x6f02884 (0x2400 bytes) 2024-09-16T14:07:46.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:46.080 INFO:teuthology.orchestra.run.smithi079.stdout:1953 write 0x1be1dbe thru 0x1be617c (0x43bf bytes) 2024-09-16T14:07:46.083 INFO:teuthology.orchestra.run.smithi079.stdout:1954 punch from 0x652e424 to 0x6535933, (0x750f bytes) 2024-09-16T14:07:46.083 INFO:teuthology.orchestra.run.smithi079.stdout:1955 read 0x4ae80e3 thru 0x4af6b4e (0xea6c bytes) 2024-09-16T14:07:46.084 INFO:teuthology.orchestra.run.smithi079.stdout:1956 read 0x5cb5fde thru 0x5cb7578 (0x159b bytes) 2024-09-16T14:07:46.102 INFO:teuthology.orchestra.run.smithi079.stdout:1961 write 0x8dcc301 thru 0x8dd7814 (0xb514 bytes) 2024-09-16T14:07:46.132 INFO:teuthology.orchestra.run.smithi079.stdout:1962 write 0x9a9648b thru 0x9a98bcc (0x2742 bytes) 2024-09-16T14:07:46.133 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:46.126+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:46.151 INFO:teuthology.orchestra.run.smithi079.stdout:1963 trunc from 0x9a98bcd to 0x2dc3bfe 2024-09-16T14:07:46.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:46.189 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:46.189 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:46.189 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:46.189 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:46.189 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:46.195 INFO:teuthology.orchestra.run.smithi079.stdout:1964 write 0x49f6dc0 thru 0x49fba9d (0x4cde bytes) 2024-09-16T14:07:46.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:46.203 INFO:teuthology.orchestra.run.smithi079.stdout:1965 writesame 0x226b4f0 thru 0x2271925 data_size 0x2(0x6436 bytes) 2024-09-16T14:07:46.220 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:46.225 INFO:teuthology.orchestra.run.smithi079.stdout:1968 trunc from 0x49fba9e to 0x178a47c 2024-09-16T14:07:46.234 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:46.226+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:46.255 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:46.284 INFO:teuthology.orchestra.run.smithi079.stdout:1972 writesame 0xd24de78 thru 0xd25b02b data_size 0x2(0xd1b4 bytes) 2024-09-16T14:07:46.288 INFO:teuthology.orchestra.run.smithi079.stdout:1973 trunc from 0xd25b02c to 0xa73aec6 2024-09-16T14:07:46.300 INFO:teuthology.orchestra.run.smithi079.stdout:1976 read 0x2bfa44f thru 0x2c06bd6 (0xc788 bytes) 2024-09-16T14:07:46.300 INFO:teuthology.orchestra.run.smithi079.stdout:1978 clone 15 order 21 su 65536 sc 10 2024-09-16T14:07:46.315 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:46.335 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:46.330+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:46.336 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:46.353 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:46.353 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9030 3.16 3:6f8d7c1d:::rbd_data.1136dd12cea0.0000000000000027:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.530175357000001, 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082377660000000005, 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9030 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818442+0000", 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7669999999999999e-06 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818452+0000", 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0763e-05 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818486+0000", 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3942000000000001e-05 2024-09-16T14:07:46.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818499+0000", 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2571e-05 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818602+0000", 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010291200000000001 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818781+0000", 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017939099999999999 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.819167+0000", 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038538 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.822000+0000", 2024-09-16T14:07:46.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028333490000000002 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826582+0000", 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045822989999999997 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826632+0000", 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9780000000000001e-05 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826677+0000", 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4611999999999999e-05 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:9863 3.11 e39/23 3:8c69c14f:::rbd_data.1136a719415e.000000000000000f:head v 39'37, mlcod=37'34)", 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.956143778000001, 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081439630000000006, 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392470+0000", 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.392472+0000", 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7239999999999999e-06 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395095+0000", 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002622943 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395116+0000", 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0784e-05 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395127+0000", 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0838e-05 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395189+0000", 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2211000000000005e-05 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.395228+0000", 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8695000000000001e-05 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.400576+0000", 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053489289999999997 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.400614+0000", 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7839000000000001e-05 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13209 3.5 3:a693b338:::rbd_data.11361fa8f1cb.000000000000016d:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.541923541999999, 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086662360000000008, 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13209 2024-09-16T14:07:46.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806692+0000", 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.345e-06 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806699+0000", 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8399999999999997e-06 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806718+0000", 2024-09-16T14:07:46.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8828e-05 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806729+0000", 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1069e-05 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.807862+0000", 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011332429999999999 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.808093+0000", 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023125700000000001 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815357+0000", 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072636539999999996 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29264 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.038252+0000", 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3103618519999998, 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0085556360000000001, 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29264 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038252+0000", 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038252+0000", 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038253+0000", 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1099999999999996e-07 2024-09-16T14:07:46.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038256+0000", 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.773e-06 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038262+0000", 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3879999999999997e-06 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038267+0000", 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7129999999999996e-06 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.043685+0000", 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054185459999999998 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.043773+0000", 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7761000000000002e-05 2024-09-16T14:07:46.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.044277+0000", 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050411900000000003 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046238+0000", 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019605540000000002 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046742+0000", 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050409499999999998 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046777+0000", 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5355999999999998e-05 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046808+0000", 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.082e-05 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29265 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.038271+0000", 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.310342994, 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091644540000000007, 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29265 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038271+0000", 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038271+0000", 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038272+0000", 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4999999999999998e-07 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038274+0000", 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6630000000000002e-06 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038280+0000", 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0929999999999999e-06 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038284+0000", 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.659e-06 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.044861+0000", 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065767610000000004 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.044940+0000", 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8992999999999994e-05 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.045436+0000", 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00049647700000000005 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046327+0000", 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089119700000000004 2024-09-16T14:07:46.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.047359+0000", 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010311649999999999 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.047405+0000", 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6896e-05 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.047436+0000", 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.01e-05 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29292 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.067757+0000", 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.2808568629999999, 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081248599999999994, 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29292 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067757+0000", 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067757+0000", 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067758+0000", 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.46e-07 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067760+0000", 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9120000000000001e-06 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067763+0000", 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8990000000000003e-06 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067766+0000", 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.072699+0000", 2024-09-16T14:07:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049330570000000002 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.072790+0000", 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1371000000000003e-05 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.073028+0000", 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023748500000000001 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.074878+0000", 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018505850000000001 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.075810+0000", 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093128299999999998 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.075848+0000", 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8531000000000001e-05 2024-09-16T14:07:46.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.075882+0000", 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4017000000000003e-05 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29293 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.067768+0000", 2024-09-16T14:07:46.388 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.2808459929999998, 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0098071449999999998, 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29293 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067768+0000", 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067768+0000", 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067768+0000", 2024-09-16T14:07:46.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1100000000000002e-07 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067771+0000", 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1660000000000001e-06 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067774+0000", 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.754e-06 2024-09-16T14:07:46.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067776+0000", 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.073562+0000", 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057857509999999996 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.073643+0000", 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1025000000000004e-05 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.074045+0000", 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000401662 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.074886+0000", 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000840886 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.077400+0000", 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002514404 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.077487+0000", 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7170999999999996e-05 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.077575+0000", 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7930999999999995e-05 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29497 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930738+0000", 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.41787602, 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081046950000000003, 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.396 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29497 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930738+0000", 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930738+0000", 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930739+0000", 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.189e-06 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930746+0000", 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4049999999999997e-06 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930764+0000", 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8462e-05 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930771+0000", 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6019999999999996e-06 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.932089+0000", 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013178529999999999 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.932243+0000", 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015424500000000001 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.932699+0000", 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00045659699999999999 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.938521+0000", 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058211369999999997 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.938645+0000", 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124413 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.938737+0000", 2024-09-16T14:07:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1926e-05 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.938843+0000", 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010586600000000001 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29498 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930779+0000", 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.4178352890000001, 2024-09-16T14:07:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011406045, 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29498 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930779+0000", 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930779+0000", 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930780+0000", 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1800000000000005e-07 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930793+0000", 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3808999999999999e-05 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930801+0000", 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6580000000000002e-06 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930808+0000", 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6630000000000004e-06 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.933274+0000", 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024665659999999999 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.933342+0000", 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7463000000000001e-05 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.933685+0000", 2024-09-16T14:07:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034355099999999999 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.939715+0000", 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006029675 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942014+0000", 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022995390000000002 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942108+0000", 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3504000000000005e-05 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942185+0000", 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6898999999999997e-05 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29499 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930814+0000", 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.4178004180000001, 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012734844, 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29499 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930814+0000", 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930814+0000", 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930814+0000", 2024-09-16T14:07:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0200000000000002e-07 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930818+0000", 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3450000000000002e-06 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930823+0000", 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9089999999999997e-06 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930828+0000", 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1180000000000002e-06 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.934399+0000", 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035717750000000001 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.934474+0000", 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4548000000000005e-05 2024-09-16T14:07:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.934895+0000", 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00042120299999999999 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942205+0000", 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007309945 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943412+0000", 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001207409 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943519+0000", 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010639100000000001 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943549+0000", 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9699000000000001e-05 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.414 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29500 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930833+0000", 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.4177811920000001, 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012909405000000001, 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29500 2024-09-16T14:07:46.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930833+0000", 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930833+0000", 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930833+0000", 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930836+0000", 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930842+0000", 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8499999999999999e-06 2024-09-16T14:07:46.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930846+0000", 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4869999999999998e-06 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.935435+0000", 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045895850000000002 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.935607+0000", 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017209199999999999 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.936141+0000", 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053332100000000001 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942215+0000", 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060738260000000001 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943609+0000", 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001394571 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943705+0000", 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5843999999999999e-05 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943742+0000", 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7421e-05 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29501 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930850+0000", 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.417764064, 2024-09-16T14:07:46.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015895756, 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29501 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930850+0000", 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930850+0000", 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930850+0000", 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0400000000000002e-07 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930854+0000", 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1360000000000001e-06 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930860+0000", 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1129999999999997e-06 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930863+0000", 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4910000000000001e-06 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.936771+0000", 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059074430000000001 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.936922+0000", 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015146800000000001 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.937692+0000", 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077007599999999996 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943845+0000", 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061529250000000001 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946682+0000", 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028372480000000001 2024-09-16T14:07:46.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946719+0000", 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6291000000000002e-05 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946746+0000", 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7161e-05 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29502 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930869+0000", 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.4177453199999999, 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016015466999999999, 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.428 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29502 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930869+0000", 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930869+0000", 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930869+0000", 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8099999999999998e-07 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930872+0000", 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7810000000000001e-06 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930878+0000", 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519999999999997e-06 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930882+0000", 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.053e-06 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.938897+0000", 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.008014429 2024-09-16T14:07:46.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.938977+0000", 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0550000000000006e-05 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.939245+0000", 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026831199999999999 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946761+0000", 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075150499999999997 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946808+0000", 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7398000000000003e-05 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946851+0000", 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3339e-05 2024-09-16T14:07:46.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946884+0000", 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3022000000000003e-05 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29503 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930886+0000", 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.4177278229999999, 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01615751, 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.434 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29503 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930886+0000", 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930886+0000", 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930887+0000", 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930889+0000", 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7250000000000002e-06 2024-09-16T14:07:46.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930895+0000", 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9800000000000003e-06 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930900+0000", 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5770000000000004e-06 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.939730+0000", 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088302420000000003 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.939793+0000", 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2865000000000002e-05 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.940155+0000", 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00036199600000000001 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946768+0000", 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066130670000000003 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946940+0000", 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017188500000000001 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947003+0000", 2024-09-16T14:07:46.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2440999999999994e-05 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947044+0000", 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1309e-05 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29504 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930904+0000", 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.4177097169999999, 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018700245000000001, 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29504 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930904+0000", 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930904+0000", 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930905+0000", 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9499999999999998e-07 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930908+0000", 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2669999999999998e-06 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930915+0000", 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1180000000000003e-06 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930919+0000", 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6500000000000002e-06 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.940623+0000", 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.009704463 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.940713+0000", 2024-09-16T14:07:46.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0035000000000001e-05 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.941250+0000", 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053693600000000001 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949459+0000", 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082083220000000005 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949518+0000", 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9864000000000001e-05 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949565+0000", 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6875000000000001e-05 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949605+0000", 2024-09-16T14:07:46.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9320000000000003e-05 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29505 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930943+0000", 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.417670778, 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018780986, 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29505 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930943+0000", 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930943+0000", 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930944+0000", 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9800000000000003e-07 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930948+0000", 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:07:46.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930978+0000", 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9590999999999998e-05 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930991+0000", 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2841e-05 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942279+0000", 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011288357000000001 2024-09-16T14:07:46.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942399+0000", 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119301 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942816+0000", 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041718499999999998 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949466+0000", 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066505569999999996 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949659+0000", 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019258900000000001 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949703+0000", 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4456000000000003e-05 2024-09-16T14:07:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949724+0000", 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1044999999999999e-05 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29506 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.931001+0000", 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.4176134089999999, 2024-09-16T14:07:46.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019857364999999998, 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29506 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931001+0000", 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931001+0000", 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931002+0000", 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.172e-06 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931007+0000", 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5310000000000001e-06 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931018+0000", 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0822000000000001e-05 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931022+0000", 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5489999999999998e-06 2024-09-16T14:07:46.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943871+0000", 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012849375999999999 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943934+0000", 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3142e-05 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.945255+0000", 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013211659999999999 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949625+0000", 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043694090000000003 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.950786+0000", 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001161406 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.950819+0000", 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2772999999999999e-05 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.950858+0000", 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9019e-05 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29507 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.931025+0000", 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.4175888210000001, 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022256030999999999, 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29507 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931025+0000", 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931025+0000", 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931026+0000", 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931029+0000", 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9569999999999998e-06 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931034+0000", 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8640000000000004e-06 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931037+0000", 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9869999999999999e-06 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.945872+0000", 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014835548 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.945971+0000", 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9042999999999993e-05 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946271+0000", 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029952400000000001 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949772+0000", 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035014349999999998 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953189+0000", 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034164740000000001 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953252+0000", 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3381999999999997e-05 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953281+0000", 2024-09-16T14:07:46.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380999999999999e-05 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29508 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.931040+0000", 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.417574584, 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022380081999999999, 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29508 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931040+0000", 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931040+0000", 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931040+0000", 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931043+0000", 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.083e-06 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931047+0000", 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4830000000000001e-06 2024-09-16T14:07:46.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931050+0000", 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.988e-06 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947094+0000", 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016043182 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947157+0000", 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3460000000000006e-05 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947653+0000", 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00049577699999999998 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949777+0000", 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00212364 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953343+0000", 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035667139999999999 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953381+0000", 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7815000000000003e-05 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953420+0000", 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8566000000000003e-05 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29509 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.931053+0000", 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.417560664, 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022698278999999998, 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29509 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931053+0000", 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931053+0000", 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:46.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931054+0000", 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1800000000000001e-07 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931056+0000", 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4169999999999999e-06 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931063+0000", 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4670000000000001e-06 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931067+0000", 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3630000000000004e-06 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.948295+0000", 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017228027 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.948388+0000", 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3108000000000004e-05 2024-09-16T14:07:46.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.948800+0000", 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041197199999999998 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.950883+0000", 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002083147 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953524+0000", 2024-09-16T14:07:46.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026409229999999999 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953714+0000", 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018944099999999999 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953752+0000", 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7996e-05 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:46.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:46.478 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:46.479 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:46.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:46.480 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:46.480 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:46.480 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:46.481 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:46.481 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:46.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:46.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:46.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:46.482 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:46.482 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:46.482 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:46.482 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:46.482 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:46.482 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:46.474+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:46.580 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:46.574+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:46.631 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:46.631 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:46.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:46.680 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:46.674+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:46.735 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:46.736 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:46.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:46.790 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:46.786+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:46.830 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T14:07:46.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T14:07:46.853 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:46.853 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:46.863 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:46.911 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:46.906+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:46.952 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone14 from 0xa73aec6 (overlap 0x178a47c) to 0x107a656 2024-09-16T14:07:46.976 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:46.976 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:46.983 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:46.978+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:46.985 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:47.081 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.081 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:47.084 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:47.078+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:47.090 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:47.111 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-16T14:07:47.130 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:47.150 INFO:teuthology.orchestra.run.smithi079.stdout:1980 write 0xe92850d thru 0xe932774 (0xa268 bytes) 2024-09-16T14:07:47.177 INFO:teuthology.orchestra.run.smithi079.stdout:1981 trunc from 0xe932775 to 0x402bb05 2024-09-16T14:07:47.185 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:47.178+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:47.194 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.194 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:47.205 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:47.213 INFO:teuthology.orchestra.run.smithi079.stdout:1982 punch from 0x2c0a96e to 0x2c1042d, (0x5abf bytes) 2024-09-16T14:07:47.213 INFO:teuthology.orchestra.run.smithi079.stdout:1984 punch from 0x251bcf to 0x26103e, (0xf46f bytes) 2024-09-16T14:07:47.213 INFO:teuthology.orchestra.run.smithi079.stdout:1985 read 0x2d38333 thru 0x2d3f350 (0x701e bytes) 2024-09-16T14:07:47.241 INFO:teuthology.orchestra.run.smithi079.stdout:1986 write 0xedcab9b thru 0xedd302e (0x8494 bytes) 2024-09-16T14:07:47.250 INFO:teuthology.orchestra.run.smithi079.stdout:1987 read 0x90c00a9 thru 0x90c41b0 (0x4108 bytes) 2024-09-16T14:07:47.250 INFO:teuthology.orchestra.run.smithi079.stdout:1988 write 0x9e63f1c thru 0x9e6a470 (0x6555 bytes) 2024-09-16T14:07:47.273 INFO:teuthology.orchestra.run.smithi079.stdout:1989 trunc from 0xedd302f to 0x61af0f9 2024-09-16T14:07:47.286 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:47.281+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:47.299 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.299 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:47.300 INFO:teuthology.orchestra.run.smithi079.stdout:1990 trunc from 0x61af0f9 to 0x3269e8f 2024-09-16T14:07:47.309 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:47.335 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:47.360 INFO:teuthology.orchestra.run.smithi079.stdout:1993 write 0x9f483a1 thru 0x9f4da98 (0x56f8 bytes) 2024-09-16T14:07:47.366 INFO:teuthology.orchestra.run.smithi079.stdout:1994 read 0x918b6fc thru 0x919acb6 (0xf5bb bytes) 2024-09-16T14:07:47.367 INFO:teuthology.orchestra.run.smithi079.stdout:1995 write 0x1d8290c thru 0x1d83a2d (0x1122 bytes) 2024-09-16T14:07:47.382 INFO:teuthology.orchestra.run.smithi079.stdout:1997 punch from 0x40aa9f4 to 0x40ba462, (0xfa6e bytes) 2024-09-16T14:07:47.383 INFO:teuthology.orchestra.run.smithi079.stdout:1998 read 0x62b466f thru 0x62c2609 (0xdf9b bytes) 2024-09-16T14:07:47.383 INFO:teuthology.orchestra.run.smithi079.stdout:1999 write 0x442830a thru 0x4435f84 (0xdc7b bytes) 2024-09-16T14:07:47.387 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:47.382+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:47.392 INFO:teuthology.orchestra.run.smithi079.stdout:2000 trunc from 0x9f4da99 to 0x8801bd3 2024-09-16T14:07:47.408 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.408 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:47.416 INFO:teuthology.orchestra.run.smithi079.stdout:2001 trunc from 0x8801bd3 to 0xaa0edda 2024-09-16T14:07:47.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:47.425 INFO:teuthology.orchestra.run.smithi079.stdout:2003 write 0x54504a8 thru 0x54576c1 (0x721a bytes) 2024-09-16T14:07:47.433 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:47.433 INFO:teuthology.orchestra.run.smithi079.stdout:2006 writesame 0x1900a5b thru 0x19015d3 data_size 0xb(0xb79 bytes) 2024-09-16T14:07:47.460 INFO:teuthology.orchestra.run.smithi079.stdout:2007 write 0xbb45bf0 thru 0xbb5574e (0xfb5f bytes) 2024-09-16T14:07:47.467 INFO:teuthology.orchestra.run.smithi079.stdout:2009 punch from 0x13e2c5b to 0x13ea0f6, (0x749b bytes) 2024-09-16T14:07:47.467 INFO:teuthology.orchestra.run.smithi079.stdout:2010 punch from 0x2fdf4dc to 0x2fe1aeb, (0x260f bytes) 2024-09-16T14:07:47.478 INFO:teuthology.orchestra.run.smithi079.stdout:2011 trunc from 0xbb5574f to 0x6739295 2024-09-16T14:07:47.487 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:47.481+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:47.496 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:47.496 INFO:teuthology.orchestra.run.smithi079.stdout:2013 read 0xc13503 thru 0xc20b28 (0xd626 bytes) 2024-09-16T14:07:47.519 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.519 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:47.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:47.541 INFO:teuthology.orchestra.run.smithi079.stdout:2014 write 0xc67cd8c thru 0xc6800ca (0x333f bytes) 2024-09-16T14:07:47.548 INFO:teuthology.orchestra.run.smithi079.stdout:2015 clone 16 order 23 su 1048576 sc 11 2024-09-16T14:07:47.589 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:47.581+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:47.626 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.626 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:47.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:47.690 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:47.685+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:47.730 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.730 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:47.739 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:47.790 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:47.782+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:47.836 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T14:07:47.837 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.837 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:47.846 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:47.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T14:07:47.890 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:47.882+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:47.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:07:47.990 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone15 from 0xc6800cb (overlap 0x3269e8f) to 0x50a974 2024-09-16T14:07:47.991 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:47.985+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:47.995 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:47.995 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:48.009 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:48.093 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:48.085+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:48.120 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:48.120 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:48.129 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:48.194 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:48.186+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:48.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:48.295 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:48.289+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:48.341 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:48.341 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:48.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:48.356 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-16T14:07:48.396 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:48.390+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:48.415 INFO:teuthology.orchestra.run.smithi079.stdout:2016 read 0x6c072e7 thru 0x6c0d842 (0x655c bytes) 2024-09-16T14:07:48.422 INFO:teuthology.orchestra.run.smithi079.stdout:2017 write 0xf9e69a thru 0xfa137a (0x2ce1 bytes) 2024-09-16T14:07:48.452 INFO:teuthology.orchestra.run.smithi079.stdout:2020 writesame 0xeaf84f3 thru 0xeafa8af data_size 0x1(0x23bd bytes) 2024-09-16T14:07:48.457 INFO:teuthology.orchestra.run.smithi079.stdout:2022 write 0x416f080 thru 0x4174f89 (0x5f0a bytes) 2024-09-16T14:07:48.470 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:48.470 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:48.477 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:48.478 INFO:teuthology.orchestra.run.smithi079.stdout:2025 read 0xe295c7b thru 0xe297e6c (0x21f2 bytes) 2024-09-16T14:07:48.478 INFO:teuthology.orchestra.run.smithi079.stdout:2027 trunc from 0xeafa8b0 to 0xe48e8e3 2024-09-16T14:07:48.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:48.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:48.497 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:48.490+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:48.559 INFO:teuthology.orchestra.run.smithi079.stdout:2028 writesame 0x5ec08a6 thru 0x5ed05e4 data_size 0x1(0xfd3f bytes) 2024-09-16T14:07:48.586 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:48.586 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:48.597 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:48.589+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:48.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:48.601 INFO:teuthology.orchestra.run.smithi079.stdout:2029 write 0x28eda25 thru 0x28f220d (0x47e9 bytes) 2024-09-16T14:07:48.620 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:48.637 INFO:teuthology.orchestra.run.smithi079.stdout:2030 trunc from 0xe48e8e3 to 0x8f08ede 2024-09-16T14:07:48.638 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:48.663 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:48.680 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:48.683 INFO:teuthology.orchestra.run.smithi079.stdout:2031 write 0xcaf05cc thru 0xcaf9c81 (0x96b6 bytes) 2024-09-16T14:07:48.698 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:48.690+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:48.701 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:48.705 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:48.705 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:48.705 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:48.705 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:48.707 INFO:teuthology.orchestra.run.smithi079.stdout:2032 trunc from 0xcaf9c82 to 0x471678 2024-09-16T14:07:48.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:48.731 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:48.767 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:48.792 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:48.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:48.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:48.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:48.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:48.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:48.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:48.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:48.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:48.800 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:48.794+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:48.806 INFO:teuthology.orchestra.run.smithi079.stdout:2033 write 0x6f99358 thru 0x6f9e79f (0x5448 bytes) 2024-09-16T14:07:48.835 INFO:teuthology.orchestra.run.smithi079.stdout:2035 writesame 0xd3b6abf thru 0xd3bceba data_size 0x51(0x63fc bytes) 2024-09-16T14:07:48.837 INFO:teuthology.orchestra.run.smithi079.stdout:2036 trunc from 0xd3bcebb to 0xdf686b1 2024-09-16T14:07:48.845 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T14:07:48.846 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:07:48.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T14:07:48.856 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:48.856 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:48.856 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:48.856 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:48.857 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:48.858 INFO:teuthology.orchestra.run.smithi079.stdout:2037 write 0x12885e thru 0x12eed4 (0x6677 bytes) 2024-09-16T14:07:48.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:48.875 INFO:teuthology.orchestra.run.smithi079.stdout:2038 write 0xe9ca3ea thru 0xe9d3d0f (0x9926 bytes) 2024-09-16T14:07:48.885 INFO:teuthology.orchestra.run.smithi079.stdout:2039 write 0x1f9010b thru 0x1f90530 (0x426 bytes) 2024-09-16T14:07:48.891 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:48.891 INFO:teuthology.orchestra.run.smithi079.stdout:2041 punch from 0xd992e18 to 0xd99f988, (0xcb70 bytes) 2024-09-16T14:07:48.891 INFO:teuthology.orchestra.run.smithi079.stdout:2042 read 0x8b2a1a8 thru 0x8b2eb52 (0x49ab bytes) 2024-09-16T14:07:48.891 INFO:teuthology.orchestra.run.smithi079.stdout:2043 read 0xafc9d74 thru 0xafd4c3a (0xaec7 bytes) 2024-09-16T14:07:48.891 INFO:teuthology.orchestra.run.smithi079.stdout:2047 read 0xb0034de thru 0xb012665 (0xf188 bytes) 2024-09-16T14:07:48.891 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:48.891 INFO:teuthology.orchestra.run.smithi079.stdout:2049 write 0xaa2158a thru 0xaa2acbf (0x9736 bytes) 2024-09-16T14:07:48.897 INFO:teuthology.orchestra.run.smithi079.stdout:2050 write 0xb76ad9f thru 0xb76d7fd (0x2a5f bytes) 2024-09-16T14:07:48.901 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:48.894+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:48.903 INFO:teuthology.orchestra.run.smithi079.stdout:2051 writesame 0x8643225 thru 0x8645b9f data_size 0x7(0x297b bytes) 2024-09-16T14:07:48.912 INFO:teuthology.orchestra.run.smithi079.stdout:2053 read 0x1cc17db thru 0x1cc7dd5 (0x65fb bytes) 2024-09-16T14:07:48.932 INFO:teuthology.orchestra.run.smithi079.stdout:2054 trunc from 0xe9d3d10 to 0x2fdc56e 2024-09-16T14:07:48.954 INFO:teuthology.orchestra.run.smithi079.stdout:2055 read 0x6c0648 thru 0x6cfde6 (0xf79f bytes) 2024-09-16T14:07:48.979 INFO:teuthology.orchestra.run.smithi079.stdout:2056 write 0xb68a962 thru 0xb6974a0 (0xcb3f bytes) 2024-09-16T14:07:48.980 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:48.980 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:48.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:48.980 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:48.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.980 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.128449111999998, 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.15070547300000001, 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.981 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847224+0000", 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:48.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847226+0000", 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.204e-06 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847235+0000", 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847250+0000", 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5662e-05 2024-09-16T14:07:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847261+0000", 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0236e-05 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847327+0000", 2024-09-16T14:07:48.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.622e-05 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.847345+0000", 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8434e-05 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.993706+0000", 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.14636041899999999 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994117+0000", 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041132499999999998 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.994428+0000", 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031060899999999999 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.995845+0000", 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014173359999999999 2024-09-16T14:07:48.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997768+0000", 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001922991 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997867+0000", 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9057999999999999e-05 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:53.997929+0000", 2024-09-16T14:07:48.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2433000000000003e-05 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.749091489999998, 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015154025999999999, 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:48.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 50 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:48.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226582+0000", 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7999999999999995e-07 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226710+0000", 2024-09-16T14:07:48.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012769999999999999 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226721+0000", 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0954e-05 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226726+0000", 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6459999999999998e-06 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:48.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238128+0000", 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011401390000000001 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238249+0000", 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121284 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.238407+0000", 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015738899999999999 2024-09-16T14:07:48.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240509+0000", 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002102923 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241459+0000", 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094996999999999996 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:48.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241621+0000", 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000161502 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241736+0000", 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011458799999999999 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:48.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.748938565000003, 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015305664, 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 51 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:48.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226734+0000", 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226735+0000", 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5700000000000001e-07 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:48.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226740+0000", 2024-09-16T14:07:48.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6460000000000001e-06 2024-09-16T14:07:48.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:48.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226748+0000", 2024-09-16T14:07:48.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:07:48.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226753+0000", 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6229999999999996e-06 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239228+0000", 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012475119999999999 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239405+0000", 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000177641 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.239559+0000", 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015368599999999999 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241393+0000", 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018340260000000001 2024-09-16T14:07:48.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241863+0000", 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047025300000000001 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241934+0000", 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0229999999999997e-05 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242040+0000", 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106568 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:49.001 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.748912171000001, 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015737872, 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 52 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226761+0000", 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226762+0000", 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.23e-07 2024-09-16T14:07:49.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226766+0000", 2024-09-16T14:07:49.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.668e-06 2024-09-16T14:07:49.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226774+0000", 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8939999999999993e-06 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.226779+0000", 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1170000000000003e-06 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240584+0000", 2024-09-16T14:07:49.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013804344 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.240709+0000", 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012524300000000001 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.241014+0000", 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030534399999999998 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242062+0000", 2024-09-16T14:07:49.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010479899999999999 2024-09-16T14:07:49.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242369+0000", 2024-09-16T14:07:49.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030710399999999999 2024-09-16T14:07:49.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242442+0000", 2024-09-16T14:07:49.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2383000000000001e-05 2024-09-16T14:07:49.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.242499+0000", 2024-09-16T14:07:49.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:07:49.009 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.010 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.010 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.010 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:49.010 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:49.010 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.204597186999997, 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015694993000000001, 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 72 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:49.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771076+0000", 2024-09-16T14:07:49.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771077+0000", 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0299999999999997e-07 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771209+0000", 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132347 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771224+0000", 2024-09-16T14:07:49.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.454e-05 2024-09-16T14:07:49.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771230+0000", 2024-09-16T14:07:49.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:07:49.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782272+0000", 2024-09-16T14:07:49.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011041455 2024-09-16T14:07:49.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782402+0000", 2024-09-16T14:07:49.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013039099999999999 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.782645+0000", 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024303500000000001 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786509+0000", 2024-09-16T14:07:49.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003863698 2024-09-16T14:07:49.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786628+0000", 2024-09-16T14:07:49.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011894499999999999 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786711+0000", 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2961999999999996e-05 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786771+0000", 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0098e-05 2024-09-16T14:07:49.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.204434458000001, 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016692631999999999, 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 73 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771239+0000", 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771240+0000", 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.569999999999999e-07 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771244+0000", 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771253+0000", 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8629999999999997e-06 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771262+0000", 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:07:49.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.783924+0000", 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012662479000000001 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784015+0000", 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0624999999999994e-05 2024-09-16T14:07:49.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784129+0000", 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113563 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787598+0000", 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034694890000000001 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787707+0000", 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109341 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787840+0000", 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132421 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787931+0000", 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1291000000000004e-05 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.204404361999998, 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017320314999999999, 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 74 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771269+0000", 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771278+0000", 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3790000000000006e-06 2024-09-16T14:07:49.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771286+0000", 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5199999999999997e-06 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771292+0000", 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2839999999999997e-06 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.784933+0000", 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013641304 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785019+0000", 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6201999999999999e-05 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785164+0000", 2024-09-16T14:07:49.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014457599999999999 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787609+0000", 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024456769999999998 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788403+0000", 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079383000000000003 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788490+0000", 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7231000000000006e-05 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788589+0000", 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8608000000000002e-05 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.204374651999998, 2024-09-16T14:07:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017489688, 2024-09-16T14:07:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.034 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 75 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:49.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771298+0000", 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771299+0000", 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3e-07 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771421+0000", 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121544 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771433+0000", 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2853e-05 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771443+0000", 2024-09-16T14:07:49.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0229999999999998e-06 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785601+0000", 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014158145 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785677+0000", 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6711999999999993e-05 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.785774+0000", 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6583999999999995e-05 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787633+0000", 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001859461 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788688+0000", 2024-09-16T14:07:49.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010546349999999999 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788747+0000", 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.923e-05 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788788+0000", 2024-09-16T14:07:49.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0771e-05 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4172.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.204218072000003, 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017898397999999999, 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4172", 2024-09-16T14:07:49.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1459479709", 2024-09-16T14:07:49.042 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 76 2024-09-16T14:07:49.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.042 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771455+0000", 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771456+0000", 2024-09-16T14:07:49.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7900000000000007e-07 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771466+0000", 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8800000000000003e-06 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771475+0000", 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1670000000000005e-06 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.771480+0000", 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.49e-06 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786862+0000", 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015381807000000001 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.786985+0000", 2024-09-16T14:07:49.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012302099999999999 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.787142+0000", 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015696500000000001 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.788157+0000", 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001014923 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789175+0000", 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010175200000000001 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789259+0000", 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4578000000000002e-05 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:06:54.789353+0000", 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4067999999999995e-05 2024-09-16T14:07:49.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:7540 3.10 3:0c0a7e4b:::rbd_data.11369af4c40c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4587520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 26.168952317999999, 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016499916, 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 7540 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806721+0000", 2024-09-16T14:07:49.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.806722+0000", 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.78e-07 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807847+0000", 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001125381 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807858+0000", 2024-09-16T14:07:49.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1219999999999999e-05 2024-09-16T14:07:49.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.807863+0000", 2024-09-16T14:07:49.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1569999999999999e-06 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808087+0000", 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00022387000000000001 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.808219+0000", 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013168199999999999 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.809044+0000", 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082524200000000001 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.814222+0000", 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005177414 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823075+0000", 2024-09-16T14:07:49.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088537470000000004 2024-09-16T14:07:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823150+0000", 2024-09-16T14:07:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4571000000000001e-05 2024-09-16T14:07:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:22.823221+0000", 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0654000000000005e-05 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9727 3.1a 3:5b31ba1a:::rbd_data.1136a719415e.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.764362494, 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.033339056999999998, 2024-09-16T14:07:49.056 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9727 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211311+0000", 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.211313+0000", 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1969999999999999e-06 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216419+0000", 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051063580000000001 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216461+0000", 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2228000000000003e-05 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216473+0000", 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1635e-05 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216560+0000", 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6745999999999999e-05 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.216730+0000", 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017044299999999999 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.217504+0000", 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077432 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.232742+0000", 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015237611 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244490+0000", 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011747506 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244586+0000", 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6106000000000004e-05 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.244650+0000", 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3906999999999996e-05 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.062 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9793 3.1a 3:5f64584f:::rbd_data.1136a719415e.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.627274072999999, 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024339903, 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9793 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348399+0000", 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.348404+0000", 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.288e-06 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352536+0000", 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041315750000000002 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352576+0000", 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0309999999999999e-05 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352590+0000", 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3524e-05 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352690+0000", 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010055700000000001 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.352873+0000", 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000182714 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.353759+0000", 2024-09-16T14:07:49.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088597700000000003 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.362898+0000", 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091386360000000003 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372631+0000", 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097337159999999999 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372700+0000", 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9016999999999993e-05 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:27.372739+0000", 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8588999999999999e-05 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14483 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701324+0000", 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.274349285, 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014535807, 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14483 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701324+0000", 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701324+0000", 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701324+0000", 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:07:49.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701327+0000", 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.055e-06 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701336+0000", 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3620000000000006e-06 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701340+0000", 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5410000000000002e-06 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.706650+0000", 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053099189999999998 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.706713+0000", 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3131000000000003e-05 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.707047+0000", 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00033344199999999998 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.711603+0000", 2024-09-16T14:07:49.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045564689999999996 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.715763+0000", 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0041601169999999996 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.715823+0000", 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9601000000000001e-05 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.715860+0000", 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6683000000000001e-05 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14484 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701345+0000", 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.274327689, 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014697109, 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.076 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14484 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701345+0000", 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701345+0000", 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701346+0000", 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3000000000000001e-07 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701349+0000", 2024-09-16T14:07:49.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0290000000000001e-06 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701356+0000", 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4580000000000004e-06 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701361+0000", 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5240000000000002e-06 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.707423+0000", 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060624650000000004 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.707496+0000", 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3059000000000006e-05 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.707777+0000", 2024-09-16T14:07:49.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028042799999999999 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.712956+0000", 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051787339999999999 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.715894+0000", 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00293882 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716009+0000", 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114549 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716042+0000", 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3513000000000003e-05 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.083 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14485 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.083 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701366+0000", 2024-09-16T14:07:49.083 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.274307094999999, 2024-09-16T14:07:49.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01479137, 2024-09-16T14:07:49.083 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.083 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.084 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14485 2024-09-16T14:07:49.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.084 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701366+0000", 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701366+0000", 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701366+0000", 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8599999999999998e-07 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701369+0000", 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7939999999999998e-06 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701376+0000", 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2710000000000003e-06 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701381+0000", 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7140000000000001e-06 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708267+0000", 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068860049999999997 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708334+0000", 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7218000000000006e-05 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708552+0000", 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021770700000000001 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.714111+0000", 2024-09-16T14:07:49.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0055590980000000002 2024-09-16T14:07:49.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716080+0000", 2024-09-16T14:07:49.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019685729999999999 2024-09-16T14:07:49.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716122+0000", 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2540999999999998e-05 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716157+0000", 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4962999999999997e-05 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14486 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701386+0000", 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.2742866, 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014881439, 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14486 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701386+0000", 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701386+0000", 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701387+0000", 2024-09-16T14:07:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9599999999999999e-07 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701390+0000", 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2509999999999999e-06 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701397+0000", 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6440000000000003e-06 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701402+0000", 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7129999999999996e-06 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708851+0000", 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0074494069999999999 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.708927+0000", 2024-09-16T14:07:49.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5766000000000006e-05 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.709252+0000", 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032530599999999998 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.714139+0000", 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048873240000000002 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716192+0000", 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020529609999999998 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716234+0000", 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1823000000000003e-05 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716268+0000", 2024-09-16T14:07:49.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3748000000000003e-05 2024-09-16T14:07:49.098 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.098 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.098 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.098 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14487 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.098 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.701407+0000", 2024-09-16T14:07:49.098 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.274266334, 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014969764, 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14487 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701407+0000", 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701407+0000", 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701407+0000", 2024-09-16T14:07:49.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8800000000000002e-07 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701434+0000", 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6961000000000002e-05 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701445+0000", 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.08e-05 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.701451+0000", 2024-09-16T14:07:49.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6899999999999997e-06 2024-09-16T14:07:49.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.710059+0000", 2024-09-16T14:07:49.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086083640000000003 2024-09-16T14:07:49.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.710327+0000", 2024-09-16T14:07:49.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026832899999999999 2024-09-16T14:07:49.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:49.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.711146+0000", 2024-09-16T14:07:49.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00081821400000000005 2024-09-16T14:07:49.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.714148+0000", 2024-09-16T14:07:49.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030022680000000002 2024-09-16T14:07:49.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716301+0000", 2024-09-16T14:07:49.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021534919999999999 2024-09-16T14:07:49.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716344+0000", 2024-09-16T14:07:49.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2395999999999999e-05 2024-09-16T14:07:49.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.716376+0000", 2024-09-16T14:07:49.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2662e-05 2024-09-16T14:07:49.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.107 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.108 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14762 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.108 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.772489+0000", 2024-09-16T14:07:49.108 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.203184022, 2024-09-16T14:07:49.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015045655, 2024-09-16T14:07:49.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.109 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.109 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14762 2024-09-16T14:07:49.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.110 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772489+0000", 2024-09-16T14:07:49.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772489+0000", 2024-09-16T14:07:49.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772490+0000", 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.049e-06 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772497+0000", 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9550000000000003e-06 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772510+0000", 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3351e-05 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772516+0000", 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1179999999999997e-06 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.779908+0000", 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073911519999999998 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.780005+0000", 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7806999999999994e-05 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.780422+0000", 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041635199999999999 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.784878+0000", 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044558790000000003 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787434+0000", 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025559060000000002 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787498+0000", 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3997000000000004e-05 2024-09-16T14:07:49.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787535+0000", 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7088999999999997e-05 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:14763 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.116 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:35.772523+0000", 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.203149822, 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015125418, 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 14763 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772523+0000", 2024-09-16T14:07:49.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772523+0000", 2024-09-16T14:07:49.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772524+0000", 2024-09-16T14:07:49.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1999999999999998e-07 2024-09-16T14:07:49.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772528+0000", 2024-09-16T14:07:49.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.301e-06 2024-09-16T14:07:49.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772545+0000", 2024-09-16T14:07:49.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7178999999999999e-05 2024-09-16T14:07:49.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.772551+0000", 2024-09-16T14:07:49.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7610000000000004e-06 2024-09-16T14:07:49.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.780943+0000", 2024-09-16T14:07:49.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083913719999999994 2024-09-16T14:07:49.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.781033+0000", 2024-09-16T14:07:49.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0451999999999994e-05 2024-09-16T14:07:49.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:49.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.781446+0000", 2024-09-16T14:07:49.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000412493 2024-09-16T14:07:49.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.784885+0000", 2024-09-16T14:07:49.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003439058 2024-09-16T14:07:49.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787594+0000", 2024-09-16T14:07:49.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027093709999999999 2024-09-16T14:07:49.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787623+0000", 2024-09-16T14:07:49.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9502e-05 2024-09-16T14:07:49.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:35.787649+0000", 2024-09-16T14:07:49.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5109e-05 2024-09-16T14:07:49.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.127 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:19008 3.1c 3:3b5e8722:::rbd_object_map.11362defb50e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.395517+0000", 2024-09-16T14:07:49.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.580155632, 2024-09-16T14:07:49.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015791547, 2024-09-16T14:07:49.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 19008 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395517+0000", 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395517+0000", 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395518+0000", 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9700000000000002e-07 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395520+0000", 2024-09-16T14:07:49.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.531e-06 2024-09-16T14:07:49.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395526+0000", 2024-09-16T14:07:49.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5489999999999997e-06 2024-09-16T14:07:49.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.395532+0000", 2024-09-16T14:07:49.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6810000000000003e-06 2024-09-16T14:07:49.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.405294+0000", 2024-09-16T14:07:49.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0097622330000000004 2024-09-16T14:07:49.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.405426+0000", 2024-09-16T14:07:49.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013181099999999999 2024-09-16T14:07:49.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:49.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.405952+0000", 2024-09-16T14:07:49.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052624799999999999 2024-09-16T14:07:49.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.408258+0000", 2024-09-16T14:07:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023066480000000001 2024-09-16T14:07:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.411197+0000", 2024-09-16T14:07:49.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029387110000000001 2024-09-16T14:07:49.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.411247+0000", 2024-09-16T14:07:49.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9938999999999997e-05 2024-09-16T14:07:49.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.411309+0000", 2024-09-16T14:07:49.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1798999999999996e-05 2024-09-16T14:07:49.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.138 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.138 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.138 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:49.139 INFO:teuthology.orchestra.run.smithi079.stdout:2057 write 0x7d6b2df thru 0x7d73fea (0x8d0c bytes) 2024-09-16T14:07:49.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:2058 writesame 0xc08e11f thru 0xc093b9e data_size 0x40(0x5a80 bytes) 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:2059 trunc from 0xc093b9f to 0xa82c669 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:2061 read 0xa4b7e50 thru 0xa4c374f (0xb900 bytes) 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:2062 read 0x4303de6 thru 0x430c268 (0x8483 bytes) 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:2064 write 0x3e19571 thru 0x3e26024 (0xcab4 bytes) 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:2066 writesame 0xecf4a70 thru 0xecf6af9 data_size 0x11(0x208a bytes) 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:2068 trunc from 0xecf6afa to 0x2f8422 2024-09-16T14:07:49.142 INFO:teuthology.orchestra.run.smithi079.stdout:2069 write 0x5f04212 thru 0x5f08e60 (0x4c4f bytes) 2024-09-16T14:07:49.143 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:49.134+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:49.158 INFO:teuthology.orchestra.run.smithi079.stdout:2071 write 0xe444121 thru 0xe4496f7 (0x55d7 bytes) 2024-09-16T14:07:49.165 INFO:teuthology.orchestra.run.smithi079.stdout:2072 punch from 0x356a0f3 to 0x356c088, (0x1f95 bytes) 2024-09-16T14:07:49.165 INFO:teuthology.orchestra.run.smithi079.stdout:2073 read 0xc7fb436 thru 0xc7fe841 (0x340c bytes) 2024-09-16T14:07:49.165 INFO:teuthology.orchestra.run.smithi079.stdout:2074 trunc from 0xe4496f8 to 0xe9e5ec9 2024-09-16T14:07:49.178 INFO:teuthology.orchestra.run.smithi079.stdout:2075 read 0xe8a6760 thru 0xe8a80a8 (0x1949 bytes) 2024-09-16T14:07:49.179 INFO:teuthology.orchestra.run.smithi079.stdout:2077 writesame 0xdaed91c thru 0xdafc3fe data_size 0x1(0xeae3 bytes) 2024-09-16T14:07:49.218 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.218 INFO:teuthology.orchestra.run.smithi079.stdout:2079 read 0x9d8cda thru 0x9dec0e (0x5f35 bytes) 2024-09-16T14:07:49.218 INFO:teuthology.orchestra.run.smithi079.stdout:2081 write 0x676e4b3 thru 0x677b5f7 (0xd145 bytes) 2024-09-16T14:07:49.222 INFO:teuthology.orchestra.run.smithi079.stdout:2082 write 0x99d62e3 thru 0x99e5456 (0xf174 bytes) 2024-09-16T14:07:49.228 INFO:teuthology.orchestra.run.smithi079.stdout:2083 write 0x41b7f4d thru 0x41baa6d (0x2b21 bytes) 2024-09-16T14:07:49.232 INFO:teuthology.orchestra.run.smithi079.stdout:2088 writesame 0xcfb17b5 thru 0xcfb4c61 data_size 0x1f(0x34ad bytes) 2024-09-16T14:07:49.234 INFO:teuthology.orchestra.run.smithi079.stdout:2089 writesame 0xbf1befc thru 0xbf24c6d data_size 0xa(0x8d72 bytes) 2024-09-16T14:07:49.240 INFO:teuthology.orchestra.run.smithi079.stdout:2091 write 0x32032af thru 0x320a1fe (0x6f50 bytes) 2024-09-16T14:07:49.240 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:49.234+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:49.246 INFO:teuthology.orchestra.run.smithi079.stdout:2092 punch from 0x1094a6b to 0x10a2774, (0xdd09 bytes) 2024-09-16T14:07:49.246 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.246 INFO:teuthology.orchestra.run.smithi079.stdout:2094 writesame 0xad2d517 thru 0xad3abd7 data_size 0x1(0xd6c1 bytes) 2024-09-16T14:07:49.267 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:49.267 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:49.267 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:49.268 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:49.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:49.289 INFO:teuthology.orchestra.run.smithi079.stdout:2095 writesame 0x3945c0b thru 0x394ef19 data_size 0x59(0x930f bytes) 2024-09-16T14:07:49.320 INFO:teuthology.orchestra.run.smithi079.stdout:2096 trunc from 0xe9e5ec9 to 0x1f24e87 2024-09-16T14:07:49.341 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:49.333+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:2098 read 0x754b87 thru 0x75e4b1 (0x992b bytes) 2024-09-16T14:07:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:2099 read 0x1a9ad50 thru 0x1a9f0fc (0x43ad bytes) 2024-09-16T14:07:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:2100 punch from 0x3ef005 to 0x3f4573, (0x556e bytes) 2024-09-16T14:07:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:2101 punch from 0xb431d2 to 0xb4d0a5, (0x9ed3 bytes) 2024-09-16T14:07:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:2103 read 0x1b7d288 thru 0x1b7f21e (0x1f97 bytes) 2024-09-16T14:07:49.345 INFO:teuthology.orchestra.run.smithi079.stdout:2104 read 0x14c6366 thru 0x14d6356 (0xfff1 bytes) 2024-09-16T14:07:49.363 INFO:teuthology.orchestra.run.smithi079.stdout:2105 writesame 0x5b78662 thru 0x5b7d222 data_size 0x29(0x4bc1 bytes) 2024-09-16T14:07:49.383 INFO:teuthology.orchestra.run.smithi079.stdout:2106 write 0x97d2d8b thru 0x97de44a (0xb6c0 bytes) 2024-09-16T14:07:49.390 INFO:teuthology.orchestra.run.smithi079.stdout:2107 write 0x7cd02ed thru 0x7cde572 (0xe286 bytes) 2024-09-16T14:07:49.396 INFO:teuthology.orchestra.run.smithi079.stdout:2109 writesame 0x4901a9 thru 0x496b46 data_size 0xb(0x699e bytes) 2024-09-16T14:07:49.397 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:49.397 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:49.397 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:49.397 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:49.398 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:49.402 INFO:teuthology.orchestra.run.smithi079.stdout:2112 writesame 0x3c5ec56 thru 0x3c68cfe data_size 0x1(0xa0a9 bytes) 2024-09-16T14:07:49.407 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:49.440 INFO:teuthology.orchestra.run.smithi079.stdout:2113 trunc from 0x97de44b to 0x7afac31 2024-09-16T14:07:49.442 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:49.434+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:49.457 INFO:teuthology.orchestra.run.smithi079.stdout:2114 read 0x472f000 thru 0x473d2f9 (0xe2fa bytes) 2024-09-16T14:07:49.471 INFO:teuthology.orchestra.run.smithi079.stdout:2115 write 0x7fe80c3 thru 0x7fe8a21 (0x95f bytes) 2024-09-16T14:07:49.476 INFO:teuthology.orchestra.run.smithi079.stdout:2117 punch from 0x61c3b09 to 0x61c4b0d, (0x1004 bytes) 2024-09-16T14:07:49.476 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.476 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.494 INFO:teuthology.orchestra.run.smithi079.stdout:2124 write 0xafd7d0e thru 0xafdb51e (0x3811 bytes) 2024-09-16T14:07:49.501 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.501 INFO:teuthology.orchestra.run.smithi079.stdout:2126 read 0x8503e9f thru 0x8511b89 (0xdceb bytes) 2024-09-16T14:07:49.502 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.502 INFO:teuthology.orchestra.run.smithi079.stdout:2130 punch from 0x997a98d to 0x997eea3, (0x4516 bytes) 2024-09-16T14:07:49.502 INFO:teuthology.orchestra.run.smithi079.stdout:2131 write 0xe17ddb thru 0xe1c8d0 (0x4af6 bytes) 2024-09-16T14:07:49.508 INFO:teuthology.orchestra.run.smithi079.stdout:2132 punch from 0x900e0f9 to 0x901a51b, (0xc422 bytes) 2024-09-16T14:07:49.508 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.508 INFO:teuthology.orchestra.run.smithi079.stdout:2135 punch from 0x5296dd1 to 0x529d14b, (0x637a bytes) 2024-09-16T14:07:49.508 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.523 INFO:teuthology.orchestra.run.smithi079.stdout:2139 trunc from 0xafdb51f to 0x1cb9974 2024-09-16T14:07:49.543 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:49.537+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:49.544 INFO:teuthology.orchestra.run.smithi079.stdout:2140 punch from 0x4bb8e1 to 0x4bc068, (0x787 bytes) 2024-09-16T14:07:49.544 INFO:teuthology.orchestra.run.smithi079.stdout:2143 punch from 0x1565091 to 0x1572cda, (0xdc49 bytes) 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:9030 3.16 3:6f8d7c1d:::rbd_data.1136dd12cea0.0000000000000027:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.735239794000002, 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082377660000000005, 2024-09-16T14:07:49.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9030 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818439+0000", 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818442+0000", 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7669999999999999e-06 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818452+0000", 2024-09-16T14:07:49.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0763e-05 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818486+0000", 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3942000000000001e-05 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818499+0000", 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2571e-05 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818602+0000", 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010291200000000001 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.818781+0000", 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017939099999999999 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.819167+0000", 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038538 2024-09-16T14:07:49.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.822000+0000", 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028333490000000002 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826582+0000", 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045822989999999997 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826632+0000", 2024-09-16T14:07:49.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9780000000000001e-05 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:25.826677+0000", 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4611999999999999e-05 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:13209 3.5 3:a693b338:::rbd_data.11361fa8f1cb.000000000000016d:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:49.564 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.746987979, 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086662360000000008, 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 13209 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:49.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806691+0000", 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806692+0000", 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.345e-06 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806699+0000", 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8399999999999997e-06 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806718+0000", 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8828e-05 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.806729+0000", 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1069e-05 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.807862+0000", 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011332429999999999 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.808093+0000", 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023125700000000001 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:34.815357+0000", 2024-09-16T14:07:49.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072636539999999996 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29264 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.038252+0000", 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.5154262889999996, 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0085556360000000001, 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29264 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038252+0000", 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038252+0000", 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038253+0000", 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1099999999999996e-07 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038256+0000", 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.773e-06 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038262+0000", 2024-09-16T14:07:49.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3879999999999997e-06 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038267+0000", 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7129999999999996e-06 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.043685+0000", 2024-09-16T14:07:49.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054185459999999998 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.043773+0000", 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7761000000000002e-05 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.044277+0000", 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050411900000000003 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046238+0000", 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019605540000000002 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046742+0000", 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050409499999999998 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046777+0000", 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5355999999999998e-05 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046808+0000", 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.082e-05 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29265 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.038271+0000", 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.5154074309999999, 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091644540000000007, 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.575 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29265 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038271+0000", 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038271+0000", 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038272+0000", 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4999999999999998e-07 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038274+0000", 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6630000000000002e-06 2024-09-16T14:07:49.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038280+0000", 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0929999999999999e-06 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.038284+0000", 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.659e-06 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.044861+0000", 2024-09-16T14:07:49.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065767610000000004 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.044940+0000", 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8992999999999994e-05 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.045436+0000", 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00049647700000000005 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.046327+0000", 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00089119700000000004 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.047359+0000", 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010311649999999999 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.047405+0000", 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6896e-05 2024-09-16T14:07:49.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.047436+0000", 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.01e-05 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29293 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.067768+0000", 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4859104299999997, 2024-09-16T14:07:49.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0098071449999999998, 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29293 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067768+0000", 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067768+0000", 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067768+0000", 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1100000000000002e-07 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067771+0000", 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1660000000000001e-06 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067774+0000", 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.754e-06 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.067776+0000", 2024-09-16T14:07:49.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.073562+0000", 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057857509999999996 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.073643+0000", 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1025000000000004e-05 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.074045+0000", 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000401662 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.074886+0000", 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000840886 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.077400+0000", 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002514404 2024-09-16T14:07:49.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.077487+0000", 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7170999999999996e-05 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.077575+0000", 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7930999999999995e-05 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29498 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930779+0000", 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.622899726, 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011406045, 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29498 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930779+0000", 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930779+0000", 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930780+0000", 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1800000000000005e-07 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930793+0000", 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3808999999999999e-05 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930801+0000", 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6580000000000002e-06 2024-09-16T14:07:49.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930808+0000", 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6630000000000004e-06 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.933274+0000", 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024665659999999999 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.933342+0000", 2024-09-16T14:07:49.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7463000000000001e-05 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.933685+0000", 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034355099999999999 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.939715+0000", 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006029675 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942014+0000", 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022995390000000002 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942108+0000", 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3504000000000005e-05 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942185+0000", 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6898999999999997e-05 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29499 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930814+0000", 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6228648549999996, 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012734844, 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29499 2024-09-16T14:07:49.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930814+0000", 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930814+0000", 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930814+0000", 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0200000000000002e-07 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930818+0000", 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3450000000000002e-06 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930823+0000", 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9089999999999997e-06 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930828+0000", 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1180000000000002e-06 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.934399+0000", 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035717750000000001 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.934474+0000", 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4548000000000005e-05 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.934895+0000", 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00042120299999999999 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942205+0000", 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.007309945 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943412+0000", 2024-09-16T14:07:49.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001207409 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943519+0000", 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010639100000000001 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943549+0000", 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9699000000000001e-05 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29500 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930833+0000", 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6228456290000004, 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012909405000000001, 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.601 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29500 2024-09-16T14:07:49.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.601 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930833+0000", 2024-09-16T14:07:49.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930833+0000", 2024-09-16T14:07:49.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930833+0000", 2024-09-16T14:07:49.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:07:49.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930836+0000", 2024-09-16T14:07:49.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:07:49.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930842+0000", 2024-09-16T14:07:49.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8499999999999999e-06 2024-09-16T14:07:49.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930846+0000", 2024-09-16T14:07:49.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4869999999999998e-06 2024-09-16T14:07:49.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.935435+0000", 2024-09-16T14:07:49.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045895850000000002 2024-09-16T14:07:49.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.935607+0000", 2024-09-16T14:07:49.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017209199999999999 2024-09-16T14:07:49.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.936141+0000", 2024-09-16T14:07:49.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053332100000000001 2024-09-16T14:07:49.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942215+0000", 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060738260000000001 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943609+0000", 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001394571 2024-09-16T14:07:49.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943705+0000", 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5843999999999999e-05 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943742+0000", 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7421e-05 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29501 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930850+0000", 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6228285009999999, 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015895756, 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29501 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930850+0000", 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930850+0000", 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930850+0000", 2024-09-16T14:07:49.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0400000000000002e-07 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930854+0000", 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1360000000000001e-06 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930860+0000", 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1129999999999997e-06 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930863+0000", 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4910000000000001e-06 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.936771+0000", 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059074430000000001 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.936922+0000", 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015146800000000001 2024-09-16T14:07:49.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.937692+0000", 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077007599999999996 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943845+0000", 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061529250000000001 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946682+0000", 2024-09-16T14:07:49.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028372480000000001 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946719+0000", 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6291000000000002e-05 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946746+0000", 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7161e-05 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29502 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930869+0000", 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6228097569999997, 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016015466999999999, 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29502 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.617 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930869+0000", 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930869+0000", 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930869+0000", 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8099999999999998e-07 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930872+0000", 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7810000000000001e-06 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930878+0000", 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519999999999997e-06 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930882+0000", 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.053e-06 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.938897+0000", 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.008014429 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.938977+0000", 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0550000000000006e-05 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.939245+0000", 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026831199999999999 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946761+0000", 2024-09-16T14:07:49.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075150499999999997 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946808+0000", 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7398000000000003e-05 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946851+0000", 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3339e-05 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946884+0000", 2024-09-16T14:07:49.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3022000000000003e-05 2024-09-16T14:07:49.623 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.623 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.623 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29503 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930886+0000", 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6227922599999998, 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.01615751, 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29503 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.624 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930886+0000", 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930886+0000", 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930887+0000", 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930889+0000", 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7250000000000002e-06 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930895+0000", 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9800000000000003e-06 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930900+0000", 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5770000000000004e-06 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.939730+0000", 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088302420000000003 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.939793+0000", 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2865000000000002e-05 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.940155+0000", 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00036199600000000001 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946768+0000", 2024-09-16T14:07:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066130670000000003 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946940+0000", 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017188500000000001 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947003+0000", 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2440999999999994e-05 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947044+0000", 2024-09-16T14:07:49.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1309e-05 2024-09-16T14:07:49.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29504 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930904+0000", 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.622774154, 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018700245000000001, 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29504 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930904+0000", 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930904+0000", 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930905+0000", 2024-09-16T14:07:49.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9499999999999998e-07 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930908+0000", 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2669999999999998e-06 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930915+0000", 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1180000000000003e-06 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930919+0000", 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6500000000000002e-06 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.940623+0000", 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.009704463 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.940713+0000", 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0035000000000001e-05 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.941250+0000", 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053693600000000001 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949459+0000", 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0082083220000000005 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949518+0000", 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9864000000000001e-05 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949565+0000", 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6875000000000001e-05 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949605+0000", 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9320000000000003e-05 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.636 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29505 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.930943+0000", 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6227352149999996, 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018780986, 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29505 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930943+0000", 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930943+0000", 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930944+0000", 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9800000000000003e-07 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930948+0000", 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930978+0000", 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9590999999999998e-05 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.930991+0000", 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2841e-05 2024-09-16T14:07:49.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942279+0000", 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011288357000000001 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942399+0000", 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119301 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.942816+0000", 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041718499999999998 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949466+0000", 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066505569999999996 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949659+0000", 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019258900000000001 2024-09-16T14:07:49.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949703+0000", 2024-09-16T14:07:49.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4456000000000003e-05 2024-09-16T14:07:49.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949724+0000", 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1044999999999999e-05 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29506 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.931001+0000", 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6226778460000002, 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019857364999999998, 2024-09-16T14:07:49.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29506 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931001+0000", 2024-09-16T14:07:49.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931001+0000", 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931002+0000", 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.172e-06 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931007+0000", 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5310000000000001e-06 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931018+0000", 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0822000000000001e-05 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931022+0000", 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5489999999999998e-06 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943871+0000", 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012849375999999999 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.943934+0000", 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3142e-05 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.945255+0000", 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013211659999999999 2024-09-16T14:07:49.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949625+0000", 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043694090000000003 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.950786+0000", 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001161406 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.950819+0000", 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2772999999999999e-05 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.950858+0000", 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9019e-05 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29507 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.650 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.931025+0000", 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6226532579999997, 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022256030999999999, 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29507 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931025+0000", 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931025+0000", 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931026+0000", 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931029+0000", 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9569999999999998e-06 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931034+0000", 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8640000000000004e-06 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931037+0000", 2024-09-16T14:07:49.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9869999999999999e-06 2024-09-16T14:07:49.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.945872+0000", 2024-09-16T14:07:49.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014835548 2024-09-16T14:07:49.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.945971+0000", 2024-09-16T14:07:49.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9042999999999993e-05 2024-09-16T14:07:49.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.946271+0000", 2024-09-16T14:07:49.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029952400000000001 2024-09-16T14:07:49.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949772+0000", 2024-09-16T14:07:49.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035014349999999998 2024-09-16T14:07:49.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953189+0000", 2024-09-16T14:07:49.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034164740000000001 2024-09-16T14:07:49.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953252+0000", 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3381999999999997e-05 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953281+0000", 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9380999999999999e-05 2024-09-16T14:07:49.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.660 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.660 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29508 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.660 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.931040+0000", 2024-09-16T14:07:49.660 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6226390210000003, 2024-09-16T14:07:49.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022380081999999999, 2024-09-16T14:07:49.661 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.661 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.661 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29508 2024-09-16T14:07:49.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.661 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931040+0000", 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931040+0000", 2024-09-16T14:07:49.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931040+0000", 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931043+0000", 2024-09-16T14:07:49.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.083e-06 2024-09-16T14:07:49.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931047+0000", 2024-09-16T14:07:49.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4830000000000001e-06 2024-09-16T14:07:49.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931050+0000", 2024-09-16T14:07:49.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.988e-06 2024-09-16T14:07:49.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947094+0000", 2024-09-16T14:07:49.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016043182 2024-09-16T14:07:49.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947157+0000", 2024-09-16T14:07:49.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3460000000000006e-05 2024-09-16T14:07:49.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.947653+0000", 2024-09-16T14:07:49.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00049577699999999998 2024-09-16T14:07:49.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.949777+0000", 2024-09-16T14:07:49.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00212364 2024-09-16T14:07:49.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953343+0000", 2024-09-16T14:07:49.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035667139999999999 2024-09-16T14:07:49.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953381+0000", 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7815000000000003e-05 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953420+0000", 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8566000000000003e-05 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.669 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:29509 3.5 3:a23e405b:::rbd_object_map.1136f306a5c5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:44.931053+0000", 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.6226251009999997, 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022698278999999998, 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 29509 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931053+0000", 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931053+0000", 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931054+0000", 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1800000000000001e-07 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931056+0000", 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4169999999999999e-06 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931063+0000", 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4670000000000001e-06 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.931067+0000", 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3630000000000004e-06 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.948295+0000", 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.017228027 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.948388+0000", 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3108000000000004e-05 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 6", 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.948800+0000", 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041197199999999998 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.950883+0000", 2024-09-16T14:07:49.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002083147 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953524+0000", 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026409229999999999 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953714+0000", 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018944099999999999 2024-09-16T14:07:49.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:44.953752+0000", 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7996e-05 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:36540 3.2 3:429e6588:::rbd_data.11361fb616f9.0000000000000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:48.730139+0000", 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.82353928799999998, 2024-09-16T14:07:49.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012174557000000001, 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 36540 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730139+0000", 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730139+0000", 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730140+0000", 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:07:49.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730161+0000", 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0905000000000001e-05 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730171+0000", 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0701e-05 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730176+0000", 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1570000000000002e-06 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730221+0000", 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5586e-05 2024-09-16T14:07:49.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730820+0000", 2024-09-16T14:07:49.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00059882800000000003 2024-09-16T14:07:49.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.742314+0000", 2024-09-16T14:07:49.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011493857 2024-09-16T14:07:49.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.682 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:36543 3.11 e54/23 3:8c0bb8e2:::rbd_object_map.11361fb616f9:head v 54'195, mlcod=54'193)", 2024-09-16T14:07:49.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:48.731927+0000", 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.82175138400000003, 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010705935999999999, 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.731927+0000", 2024-09-16T14:07:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.731927+0000", 2024-09-16T14:07:49.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.731928+0000", 2024-09-16T14:07:49.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.096e-06 2024-09-16T14:07:49.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.731936+0000", 2024-09-16T14:07:49.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6059999999999997e-06 2024-09-16T14:07:49.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.731947+0000", 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1494999999999999e-05 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.731953+0000", 2024-09-16T14:07:49.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:07:49.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.732023+0000", 2024-09-16T14:07:49.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9799e-05 2024-09-16T14:07:49.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.732044+0000", 2024-09-16T14:07:49.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1251999999999999e-05 2024-09-16T14:07:49.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:49.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.739638+0000", 2024-09-16T14:07:49.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075934649999999998 2024-09-16T14:07:49.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.742633+0000", 2024-09-16T14:07:49.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029954719999999999 2024-09-16T14:07:49.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.689 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:36799 3.5 3:a4463638:::rbd_data.11369f5c332a.000000000000000f:12 [read 647168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:48.734079+0000", 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.81959986299999998, 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0081688839999999995, 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:49.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 36799 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.734079+0000", 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:49.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.734079+0000", 2024-09-16T14:07:49.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:49.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:49.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.734079+0000", 2024-09-16T14:07:49.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0100000000000005e-07 2024-09-16T14:07:49.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:49.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.734081+0000", 2024-09-16T14:07:49.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1969999999999999e-06 2024-09-16T14:07:49.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.693 INFO:teuthology.orchestra.run.smithi079.stdout:2145 writesame 0x3bcbb0b thru 0x3bdb9e6 data_size 0x6(0xfedc bytes) 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.734100+0000", 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8462e-05 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.734104+0000", 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1139999999999999e-06 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.742084+0000", 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079799699999999994 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.742171+0000", 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6860000000000005e-05 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.742248+0000", 2024-09-16T14:07:49.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6779999999999993e-05 2024-09-16T14:07:49.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.696 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:49.696 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:49.696 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:49.696 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:2147 write 0x8f64897 thru 0x8f649e6 (0x150 bytes) 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:2148 write 0x6f01889 thru 0x6f09aaa (0x8222 bytes) 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:2150 punch from 0x5f67124 to 0x5f6fe2e, (0x8d0a bytes) 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:2152 read 0x3de5fa1 thru 0x3df056a (0xa5ca bytes) 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:2154 trunc from 0x8f649e7 to 0x8136f21 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.699 INFO:teuthology.orchestra.run.smithi079.stdout:2156 punch from 0x1539665 to 0x1546f6d, (0xd908 bytes) 2024-09-16T14:07:49.700 INFO:teuthology.orchestra.run.smithi079.stdout:2157 write 0x238aa83 thru 0x238aac0 (0x3e bytes) 2024-09-16T14:07:49.700 INFO:teuthology.orchestra.run.smithi079.stdout:2158 trunc from 0x8136f21 to 0x414755a 2024-09-16T14:07:49.700 INFO:teuthology.orchestra.run.smithi079.stdout:2161 punch from 0x40e7e6f to 0x40f3e53, (0xbfe4 bytes) 2024-09-16T14:07:49.700 INFO:teuthology.orchestra.run.smithi079.stdout:2162 write 0x8ccbd37 thru 0x8cd31c4 (0x748e bytes) 2024-09-16T14:07:49.700 INFO:teuthology.orchestra.run.smithi079.stdout:2163 punch from 0x70bce2f to 0x70cc99b, (0xfb6c bytes) 2024-09-16T14:07:49.700 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.700 INFO:teuthology.orchestra.run.smithi079.stdout:2167 writesame 0x1f5ca7e thru 0x1f6c4b1 data_size 0x86(0xfa34 bytes) 2024-09-16T14:07:49.710 INFO:teuthology.orchestra.run.smithi079.stdout:2171 writesame 0xd3b2c63 thru 0xd3bf3ae data_size 0xa(0xc74c bytes) 2024-09-16T14:07:49.710 INFO:teuthology.orchestra.run.smithi079.stdout:2172 read 0x689ea53 thru 0x68a8300 (0x98ae bytes) 2024-09-16T14:07:49.710 INFO:teuthology.orchestra.run.smithi079.stdout:2173 punch from 0x7756a8a to 0x775fe8b, (0x9401 bytes) 2024-09-16T14:07:49.710 INFO:teuthology.orchestra.run.smithi079.stdout:2174 punch from 0x524019e to 0x524a714, (0xa576 bytes) 2024-09-16T14:07:49.710 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:49.705+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:49.726 INFO:teuthology.orchestra.run.smithi079.stdout:2175 trunc from 0xd3bf3af to 0x56d73c7 2024-09-16T14:07:49.750 INFO:teuthology.orchestra.run.smithi079.stdout:2176 punch from 0x1465d22 to 0x1470a50, (0xad2e bytes) 2024-09-16T14:07:49.750 INFO:teuthology.orchestra.run.smithi079.stdout:2177 read 0x5323e5a thru 0x53304b7 (0xc65e bytes) 2024-09-16T14:07:49.750 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.750 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:49.751 INFO:teuthology.orchestra.run.smithi079.stdout:2184 write 0x31d2525 thru 0x31dc585 (0xa061 bytes) 2024-09-16T14:07:49.753 INFO:teuthology.orchestra.run.smithi079.stdout:2185 clone 17 order 25 su 262144 sc 14 2024-09-16T14:07:49.801 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:49.794+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:49.815 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:49.815 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:49.815 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:49.815 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:49.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:07:49.857 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T14:07:49.871 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T14:07:49.918 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:49.913+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:49.951 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:49.951 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:49.951 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:49.951 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:49.952 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:49.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:07:50.002 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:49.994+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:50.026 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone16 from 0x56d73c7 (overlap 0x2f8422) to 0x2142e4 2024-09-16T14:07:50.072 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:50.072 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4241.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.589960154000003, 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.095465191000000005, 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4241", 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2311749971", 2024-09-16T14:07:50.073 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479095+0000", 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479100+0000", 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9840000000000001e-06 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479107+0000", 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7920000000000004e-06 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479120+0000", 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3195e-05 2024-09-16T14:07:50.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479131+0000", 2024-09-16T14:07:50.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1416999999999999e-05 2024-09-16T14:07:50.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479210+0000", 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8223e-05 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.479224+0000", 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.436e-05 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570355+0000", 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.091131223999999997 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570668+0000", 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031290000000000002 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.570990+0000", 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032147600000000001 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574367+0000", 2024-09-16T14:07:50.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033769680000000002 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574482+0000", 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115021 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574528+0000", 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6035999999999998e-05 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:00.574560+0000", 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2595000000000002e-05 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12319 3.1d 3:bbef0c8d:::rbd_data.11364aaed44f.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.164716639000002, 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031934152, 2024-09-16T14:07:50.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12319 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904339+0000", 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904340+0000", 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.742e-06 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904350+0000", 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6719999999999999e-06 2024-09-16T14:07:50.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904372+0000", 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1787999999999999e-05 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904381+0000", 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9099999999999994e-06 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904477+0000", 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6571999999999996e-05 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.904631+0000", 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000153311 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.912219+0000", 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075880990000000001 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936131+0000", 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023912255 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936208+0000", 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7464000000000002e-05 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936244+0000", 2024-09-16T14:07:50.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5369e-05 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.936273+0000", 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8969999999999999e-05 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:12330 3.d 3:b20fa72b:::rbd_data.11364aaed44f.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.090237165000001, 2024-09-16T14:07:50.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.032907032000000003, 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 12330 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978818+0000", 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978820+0000", 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9860000000000001e-06 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978831+0000", 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1457999999999999e-05 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978855+0000", 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3201e-05 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978864+0000", 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6299999999999993e-06 2024-09-16T14:07:50.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.978954+0000", 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9420999999999996e-05 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.979080+0000", 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012617800000000001 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:50.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:30.986286+0000", 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072064429999999999 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011593+0000", 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025306443000000001 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011654+0000", 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0764999999999998e-05 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011696+0000", 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2330999999999998e-05 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:31.011725+0000", 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9176e-05 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.092 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15974 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890663753, 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022185745, 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15974 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178391+0000", 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178392+0000", 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178393+0000", 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.308e-06 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178396+0000", 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4209999999999999e-06 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178399+0000", 2024-09-16T14:07:50.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3120000000000001e-06 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200491+0000", 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022092166999999999 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200529+0000", 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8380999999999997e-05 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200577+0000", 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7868000000000003e-05 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15976 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890644382, 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022266966999999999, 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15976 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:50.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178411+0000", 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178412+0000", 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178416+0000", 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4489999999999999e-06 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178418+0000", 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2900000000000001e-06 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200596+0000", 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022178192999999999 2024-09-16T14:07:50.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200633+0000", 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6792999999999998e-05 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200678+0000", 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4743999999999997e-05 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15977 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890634618, 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022367227999999999, 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15977 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178421+0000", 2024-09-16T14:07:50.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.84e-07 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178422+0000", 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.257e-06 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178426+0000", 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178429+0000", 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4200000000000001e-06 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200694+0000", 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022265367000000001 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200735+0000", 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0908000000000002e-05 2024-09-16T14:07:50.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200788+0000", 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2787999999999997e-05 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15978 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890623973, 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022476471000000001, 2024-09-16T14:07:50.106 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15978 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178431+0000", 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178432+0000", 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0800000000000001e-07 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178435+0000", 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7249999999999999e-06 2024-09-16T14:07:50.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178439+0000", 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.507e-06 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178441+0000", 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5579999999999999e-06 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200803+0000", 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022361322999999999 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200841+0000", 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8353999999999999e-05 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200908+0000", 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6696e-05 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15980 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890599837, 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022615139999999999, 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15980 2024-09-16T14:07:50.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178455+0000", 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178456+0000", 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8299999999999998e-07 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178457+0000", 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.271e-06 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178460+0000", 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.455e-06 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178463+0000", 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.306e-06 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.200933+0000", 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022470054999999999 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201000+0000", 2024-09-16T14:07:50.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7117000000000001e-05 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201071+0000", 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0653000000000003e-05 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15982 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:50.115 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890580323, 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022793758000000001, 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15982 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178475+0000", 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0499999999999999e-07 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178479+0000", 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.433e-06 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178482+0000", 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.63e-06 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178485+0000", 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4789999999999999e-06 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201098+0000", 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022612818 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201185+0000", 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7886999999999994e-05 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201269+0000", 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3206000000000003e-05 2024-09-16T14:07:50.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15983 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890567994, 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022918754, 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.120 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15983 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178487+0000", 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178488+0000", 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178489+0000", 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.373e-06 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178492+0000", 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5609999999999999e-06 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178495+0000", 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3499999999999999e-06 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201297+0000", 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022801920999999999 2024-09-16T14:07:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201345+0000", 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8643000000000002e-05 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201406+0000", 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0571e-05 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.124 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15984 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890552702000001, 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023018235000000001, 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.125 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15984 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178503+0000", 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4499999999999997e-07 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178504+0000", 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.37e-06 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178508+0000", 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4879999999999999e-06 2024-09-16T14:07:50.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178510+0000", 2024-09-16T14:07:50.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3779999999999999e-06 2024-09-16T14:07:50.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201424+0000", 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022913830999999999 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201461+0000", 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7002000000000002e-05 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201521+0000", 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9620999999999997e-05 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.129 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15986 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890510127000001, 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023085504, 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15986 2024-09-16T14:07:50.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178545+0000", 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178546+0000", 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7199999999999999e-07 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178548+0000", 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.097e-06 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178554+0000", 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2820000000000002e-06 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178558+0000", 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.878e-06 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201540+0000", 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022982318000000002 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201575+0000", 2024-09-16T14:07:50.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5138999999999997e-05 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201631+0000", 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5217999999999999e-05 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15988 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:50.134 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890475995999999, 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023229864999999999, 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15988 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178579+0000", 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178580+0000", 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1300000000000001e-07 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178582+0000", 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178589+0000", 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178594+0000", 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1459999999999998e-06 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201654+0000", 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023060664000000002 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201725+0000", 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0451999999999996e-05 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201809+0000", 2024-09-16T14:07:50.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4395999999999996e-05 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15989 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 860160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890457039999999, 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023391981999999999, 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15989 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178598+0000", 2024-09-16T14:07:50.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178599+0000", 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9200000000000001e-07 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178601+0000", 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.232e-06 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178608+0000", 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9789999999999996e-06 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178612+0000", 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0890000000000002e-06 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201835+0000", 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023222702000000001 2024-09-16T14:07:50.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201905+0000", 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0425000000000004e-05 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.201990+0000", 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5062999999999995e-05 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15990 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890438868, 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023564649, 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15990 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178616+0000", 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178617+0000", 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.63e-07 2024-09-16T14:07:50.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178619+0000", 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.097e-06 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178626+0000", 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6320000000000002e-06 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178629+0000", 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8480000000000003e-06 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202017+0000", 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023387811000000001 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202111+0000", 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4055999999999995e-05 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202181+0000", 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9741999999999997e-05 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15992 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890398629, 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.02370862, 2024-09-16T14:07:50.148 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15992 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178657+0000", 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6600000000000002e-07 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178659+0000", 2024-09-16T14:07:50.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3379999999999999e-06 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178666+0000", 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7050000000000002e-06 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178670+0000", 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6789999999999998e-06 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202206+0000", 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023536148999999999 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202248+0000", 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2539e-05 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202365+0000", 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011674400000000001 2024-09-16T14:07:50.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15993 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890381315999999, 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024088607000000001, 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15993 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178674+0000", 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999999e-07 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178677+0000", 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2180000000000002e-06 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178684+0000", 2024-09-16T14:07:50.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0770000000000002e-06 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178694+0000", 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0241000000000001e-05 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202580+0000", 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023885937999999999 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202667+0000", 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7425999999999999e-05 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202763+0000", 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5235000000000006e-05 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.157 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:15995 3.11 3:88017a08:::rbd_data.11364aaed44f.0000000000000003:c [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.890296472999999, 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024187608999999999, 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 15995 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178759+0000", 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2800000000000003e-07 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178761+0000", 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6810000000000001e-06 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178765+0000", 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4259999999999996e-06 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.178768+0000", 2024-09-16T14:07:50.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6740000000000001e-06 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202791+0000", 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024023224999999999 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202865+0000", 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4206000000000001e-05 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:36.202946+0000", 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1069000000000006e-05 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:36543 3.11 3:8c0bb8e2:::rbd_object_map.11361fb616f9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:48.729105+0000", 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.3399505309999999, 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.022358922999999999, 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.162 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 36543 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.729105+0000", 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.729105+0000", 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.729105+0000", 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3399999999999999e-07 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.729108+0000", 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1149999999999998e-06 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.729114+0000", 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9800000000000003e-06 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.729118+0000", 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.641e-06 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.729229+0000", 2024-09-16T14:07:50.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111053 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.729328+0000", 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8516000000000003e-05 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.730931+0000", 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001602947 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:50.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.736647+0000", 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057160170000000003 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.751166+0000", 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014519890000000001 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.751219+0000", 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2948999999999997e-05 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:48.751464+0000", 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000244181 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:37239 3.11 3:8a471bd5:::rbd_data.11361fb616f9.000000000000001f:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:49.180918+0000", 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.88813708599999996, 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.036796560999999998, 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:50.168 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 37239 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.180918+0000", 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.180918+0000", 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.180919+0000", 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0950000000000001e-06 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.180925+0000", 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8119999999999996e-06 2024-09-16T14:07:50.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.180941+0000", 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5756e-05 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.180950+0000", 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4099999999999997e-06 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.181002+0000", 2024-09-16T14:07:50.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2024999999999997e-05 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.181097+0000", 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4640000000000002e-05 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.188732+0000", 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0076354750000000001 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.217583+0000", 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028850351 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.217598+0000", 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5447e-05 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.217666+0000", 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7633999999999997e-05 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:50.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:50.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:49.217715+0000", 2024-09-16T14:07:50.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8915999999999997e-05 2024-09-16T14:07:50.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:50.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:50.174 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:50.175 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:50.176 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:50.170+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:50.224 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-16T14:07:50.247 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.247 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.276 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:50.270+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:50.276 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:50.277 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:50.277 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:50.281 INFO:teuthology.orchestra.run.smithi079.stdout:2191 writesame 0xe71ab54 thru 0xe726c41 data_size 0x37(0xc0ee bytes) 2024-09-16T14:07:50.286 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:50.290 INFO:teuthology.orchestra.run.smithi079.stdout:2192 read 0x7f58766 thru 0x7f5e4d2 (0x5d6d bytes) 2024-09-16T14:07:50.290 INFO:teuthology.orchestra.run.smithi079.stdout:2194 read 0x81ac213 thru 0x81ad242 (0x1030 bytes) 2024-09-16T14:07:50.291 INFO:teuthology.orchestra.run.smithi079.stdout:2195 write 0xaeff2c1 thru 0xaf03dad (0x4aed bytes) 2024-09-16T14:07:50.301 INFO:teuthology.orchestra.run.smithi079.stdout:2196 read 0x649bf9d thru 0x64abac5 (0xfb29 bytes) 2024-09-16T14:07:50.301 INFO:teuthology.orchestra.run.smithi079.stdout:2197 write 0x581f058 thru 0x582ebc0 (0xfb69 bytes) 2024-09-16T14:07:50.309 INFO:teuthology.orchestra.run.smithi079.stdout:2198 read 0x3eef55e thru 0x3ef7046 (0x7ae9 bytes) 2024-09-16T14:07:50.310 INFO:teuthology.orchestra.run.smithi079.stdout:2199 read 0x68a8926 thru 0x68a9af3 (0x11ce bytes) 2024-09-16T14:07:50.310 INFO:teuthology.orchestra.run.smithi079.stdout:2200 writesame 0x876b489 thru 0x87736d5 data_size 0x1(0x824d bytes) 2024-09-16T14:07:50.345 INFO:teuthology.orchestra.run.smithi079.stdout:2201 write 0x8e8a3da thru 0x8e96edc (0xcb03 bytes) 2024-09-16T14:07:50.356 INFO:teuthology.orchestra.run.smithi079.stdout:2203 write 0x1814427 thru 0x181ab6c (0x6746 bytes) 2024-09-16T14:07:50.364 INFO:teuthology.orchestra.run.smithi079.stdout:2204 read 0x6b08bb0 thru 0x6b0f80b (0x6c5c bytes) 2024-09-16T14:07:50.365 INFO:teuthology.orchestra.run.smithi079.stdout:2205 write 0x9deba8 thru 0x9decdf (0x138 bytes) 2024-09-16T14:07:50.372 INFO:teuthology.orchestra.run.smithi079.stdout:2206 write 0x12030f4 thru 0x1209581 (0x648e bytes) 2024-09-16T14:07:50.374 INFO:teuthology.orchestra.run.smithi079.stdout:2208 write 0x39ac0dd thru 0x39acbd9 (0xafd bytes) 2024-09-16T14:07:50.376 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:50.370+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:50.382 INFO:teuthology.orchestra.run.smithi079.stdout:2209 read 0x26386a5 thru 0x264052b (0x7e87 bytes) 2024-09-16T14:07:50.383 INFO:teuthology.orchestra.run.smithi079.stdout:2210 read 0x11107fd thru 0x1114c0a (0x440e bytes) 2024-09-16T14:07:50.384 INFO:teuthology.orchestra.run.smithi079.stdout:2211 writesame 0x18bac2f thru 0x18c071c data_size 0x2(0x5aee bytes) 2024-09-16T14:07:50.385 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:50.385 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:50.394 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:50.403 INFO:teuthology.orchestra.run.smithi079.stdout:truncating to largest ever: 0xee513fa 2024-09-16T14:07:50.404 INFO:teuthology.orchestra.run.smithi079.stdout:2213 trunc from 0xe726c42 to 0xee513fa 2024-09-16T14:07:50.420 INFO:teuthology.orchestra.run.smithi079.stdout:2214 trunc from 0xee513fa to 0xd2f577c 2024-09-16T14:07:50.451 INFO:teuthology.orchestra.run.smithi079.stdout:2215 punch from 0xa90ae0b to 0xa90e21e, (0x3413 bytes) 2024-09-16T14:07:50.464 INFO:teuthology.orchestra.run.smithi079.stdout:2216 write 0xe01c128 thru 0xe02395e (0x7837 bytes) 2024-09-16T14:07:50.467 INFO:teuthology.orchestra.run.smithi079.stdout:2218 read 0xb81d5a7 thru 0xb82498d (0x73e7 bytes) 2024-09-16T14:07:50.468 INFO:teuthology.orchestra.run.smithi079.stdout:2221 punch from 0xa95b381 to 0xa95f3f2, (0x4071 bytes) 2024-09-16T14:07:50.468 INFO:teuthology.orchestra.run.smithi079.stdout:2222 writesame 0x640da37 thru 0x64148e9 data_size 0x1(0x6eb3 bytes) 2024-09-16T14:07:50.478 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:50.473+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:50.491 INFO:teuthology.orchestra.run.smithi079.stdout:2223 read 0xd64858e thru 0xd64bb39 (0x35ac bytes) 2024-09-16T14:07:50.492 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.492 INFO:teuthology.orchestra.run.smithi079.stdout:2225 punch from 0xbcad8fe to 0xbcae95f, (0x1061 bytes) 2024-09-16T14:07:50.492 INFO:teuthology.orchestra.run.smithi079.stdout:2226 writesame 0xcafb5ec thru 0xcafce44 data_size 0x1(0x1859 bytes) 2024-09-16T14:07:50.497 INFO:teuthology.orchestra.run.smithi079.stdout:2227 read 0xb557d36 thru 0xb56320c (0xb4d7 bytes) 2024-09-16T14:07:50.498 INFO:teuthology.orchestra.run.smithi079.stdout:2229 punch from 0x660fa44 to 0x6611dfb, (0x23b7 bytes) 2024-09-16T14:07:50.510 INFO:teuthology.orchestra.run.smithi079.stdout:2230 writesame 0xe0dc968 thru 0xe0e46a3 data_size 0x46(0x7d3c bytes) 2024-09-16T14:07:50.513 INFO:teuthology.orchestra.run.smithi079.stdout:2231 writesame 0xd4430af thru 0xd44dbf4 data_size 0xb(0xab46 bytes) 2024-09-16T14:07:50.519 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.519 INFO:teuthology.orchestra.run.smithi079.stdout:2233 read 0x19081ef thru 0x1911af4 (0x9906 bytes) 2024-09-16T14:07:50.520 INFO:teuthology.orchestra.run.smithi079.stdout:2234 punch from 0xc2f6039 to 0xc300441, (0xa408 bytes) 2024-09-16T14:07:50.520 INFO:teuthology.orchestra.run.smithi079.stdout:2235 punch from 0xbdf0ca8 to 0xbdf4bfd, (0x3f55 bytes) 2024-09-16T14:07:50.520 INFO:teuthology.orchestra.run.smithi079.stdout:2236 read 0x3de0de0 thru 0x3ded360 (0xc581 bytes) 2024-09-16T14:07:50.538 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:918 3.6 3:66edd6af:::rbd_object_map.1136e0d3367d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:50.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.801925079999997, 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015774501, 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 918 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730964+0000", 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730965+0000", 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730967+0000", 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:07:50.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730971+0000", 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5080000000000001e-06 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.730973+0000", 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.091e-06 2024-09-16T14:07:50.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.739992+0000", 2024-09-16T14:07:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090191430000000003 2024-09-16T14:07:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740162+0000", 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016968899999999999 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.740834+0000", 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067256100000000001 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.744120+0000", 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032859370000000001 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746643+0000", 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025232050000000002 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746700+0000", 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6251000000000001e-05 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:16.746739+0000", 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9201e-05 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1141 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.196050272999997, 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014042171000000001, 2024-09-16T14:07:50.545 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1141 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336839+0000", 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336840+0000", 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2600000000000002e-07 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336844+0000", 2024-09-16T14:07:50.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7419999999999999e-06 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336850+0000", 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336855+0000", 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4939999999999997e-06 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347139+0000", 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010284331000000001 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347252+0000", 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011294300000000001 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347714+0000", 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00046169700000000001 2024-09-16T14:07:50.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350720+0000", 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030062359999999998 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350774+0000", 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3415999999999998e-05 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350837+0000", 2024-09-16T14:07:50.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3151999999999994e-05 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350881+0000", 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4591e-05 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1142 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:50.551 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.196030022999999, 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016069268000000001, 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1142 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336859+0000", 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336860+0000", 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336866+0000", 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5600000000000001e-06 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336872+0000", 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9410000000000001e-06 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336877+0000", 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.172e-06 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348737+0000", 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011860239999999999 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348829+0000", 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1657999999999996e-05 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349140+0000", 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031145499999999998 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352798+0000", 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036581550000000002 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352815+0000", 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6883000000000001e-05 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352876+0000", 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0871e-05 2024-09-16T14:07:50.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352929+0000", 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2781000000000002e-05 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1143 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:50.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.196008624000001, 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018939362000000001, 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1143 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:50.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.44e-07 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336884+0000", 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336891+0000", 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336895+0000", 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0820000000000001e-06 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349584+0000", 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012689283000000001 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349708+0000", 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123793 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350151+0000", 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044259900000000001 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352993+0000", 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028426979999999998 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355720+0000", 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002726642 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355762+0000", 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2116000000000001e-05 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355820+0000", 2024-09-16T14:07:50.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8118999999999997e-05 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.195990518000002, 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:07:50.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:07:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:07:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:07:50.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:07:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:07:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:07:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:50.571 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.195968121, 2024-09-16T14:07:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:07:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:07:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:07:50.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:07:50.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:07:50.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.195361409999997, 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:07:50.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.584 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.195287082999997, 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:07:50.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:07:50.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:07:50.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:07:50.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1150 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:50.590 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.187937163000001, 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017570243999999999, 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1150 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344952+0000", 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344954+0000", 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4959999999999999e-06 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344964+0000", 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8719999999999997e-06 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344980+0000", 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6622000000000001e-05 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.344988+0000", 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6879999999999999e-06 2024-09-16T14:07:50.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354864+0000", 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098759160000000002 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354938+0000", 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4179999999999998e-05 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355250+0000", 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000311973 2024-09-16T14:07:50.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.361925+0000", 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066749859999999999 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362285+0000", 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036007999999999998 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362462+0000", 2024-09-16T14:07:50.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017646200000000001 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362523+0000", 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1151 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.187448412999998, 2024-09-16T14:07:50.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017278714000000001, 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1151 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345441+0000", 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345442+0000", 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.42e-06 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345450+0000", 2024-09-16T14:07:50.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2420000000000003e-06 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345461+0000", 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1395e-05 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.345467+0000", 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3600000000000001e-06 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355848+0000", 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010380162 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355940+0000", 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2629999999999999e-05 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356352+0000", 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041188100000000001 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362037+0000", 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005684911 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362545+0000", 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050792899999999995 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362636+0000", 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1479999999999998e-05 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362720+0000", 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3304000000000001e-05 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1154 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.185067187999998, 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015133097999999999, 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.602 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1154 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347822+0000", 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347823+0000", 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347832+0000", 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347854+0000", 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2086000000000002e-05 2024-09-16T14:07:50.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.347864+0000", 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356827+0000", 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089628060000000002 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356897+0000", 2024-09-16T14:07:50.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0494000000000006e-05 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357171+0000", 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027356199999999998 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362741+0000", 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055696920000000002 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362867+0000", 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012610799999999999 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362922+0000", 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4911000000000003e-05 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362955+0000", 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3810999999999999e-05 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1155 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.184780717000002, 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014975097999999999, 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1155 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348109+0000", 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348110+0000", 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6700000000000002e-07 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348114+0000", 2024-09-16T14:07:50.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2560000000000004e-06 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348129+0000", 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4725e-05 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.348134+0000", 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.152e-06 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357588+0000", 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094536949999999998 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357666+0000", 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8664000000000004e-05 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357994+0000", 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032747700000000001 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362762+0000", 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047681829999999996 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363017+0000", 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025504500000000002 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363056+0000", 2024-09-16T14:07:50.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9357000000000003e-05 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363084+0000", 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7577e-05 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1161 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.181808115000003, 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017258941999999999, 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1161 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351081+0000", 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351083+0000", 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2270000000000001e-06 2024-09-16T14:07:50.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351089+0000", 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.178e-06 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351108+0000", 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9457e-05 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351116+0000", 2024-09-16T14:07:50.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6499999999999996e-06 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360527+0000", 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094112959999999996 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.360645+0000", 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011791299999999999 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.361269+0000", 2024-09-16T14:07:50.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062395199999999999 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365240+0000", 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039707470000000002 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368249+0000", 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030092000000000001 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368302+0000", 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3146999999999998e-05 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368340+0000", 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8175e-05 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.620 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1162 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.180080371000003, 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017877857, 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1162 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352809+0000", 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352810+0000", 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.234e-06 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352818+0000", 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352831+0000", 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2839e-05 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352838+0000", 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1790000000000002e-06 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363259+0000", 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010421616 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363370+0000", 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111134 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.363713+0000", 2024-09-16T14:07:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034221700000000002 2024-09-16T14:07:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.367148+0000", 2024-09-16T14:07:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034350029999999998 2024-09-16T14:07:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370616+0000", 2024-09-16T14:07:50.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034687419999999999 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370660+0000", 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3365000000000002e-05 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370687+0000", 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7121999999999999e-05 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1164 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.179004345999999, 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016868919, 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1164 2024-09-16T14:07:50.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353885+0000", 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353886+0000", 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2130000000000001e-06 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353892+0000", 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1299999999999998e-06 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353903+0000", 2024-09-16T14:07:50.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0821e-05 2024-09-16T14:07:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353909+0000", 2024-09-16T14:07:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2909999999999998e-06 2024-09-16T14:07:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364201+0000", 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010292318 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364295+0000", 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3667999999999998e-05 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.364657+0000", 2024-09-16T14:07:50.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036279999999999998 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.367209+0000", 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025513250000000001 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370697+0000", 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034881600000000001 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370729+0000", 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1977999999999998e-05 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370754+0000", 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5215000000000002e-05 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1166 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.176756761999997, 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014686309999999999, 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1166 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356133+0000", 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9400000000000004e-07 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356137+0000", 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.473e-06 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356161+0000", 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3968000000000002e-05 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.356167+0000", 2024-09-16T14:07:50.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5840000000000002e-06 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365282+0000", 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091153269999999995 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365369+0000", 2024-09-16T14:07:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7064000000000005e-05 2024-09-16T14:07:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.365709+0000", 2024-09-16T14:07:50.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00034009900000000001 2024-09-16T14:07:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.368388+0000", 2024-09-16T14:07:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002679393 2024-09-16T14:07:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370762+0000", 2024-09-16T14:07:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023732319999999999 2024-09-16T14:07:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370793+0000", 2024-09-16T14:07:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1550000000000001e-05 2024-09-16T14:07:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.370819+0000", 2024-09-16T14:07:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5826e-05 2024-09-16T14:07:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1325 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-16T14:07:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:50.643 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.813148595000001, 2024-09-16T14:07:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014200757, 2024-09-16T14:07:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.644 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1325 2024-09-16T14:07:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:50.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719741+0000", 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719742+0000", 2024-09-16T14:07:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3459999999999999e-06 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719750+0000", 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6569999999999997e-06 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719777+0000", 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7370000000000001e-05 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.719785+0000", 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2190000000000004e-06 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728088+0000", 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083028860000000006 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728229+0000", 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014092799999999999 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.728704+0000", 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047427599999999998 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.729579+0000", 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00087516299999999998 2024-09-16T14:07:50.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733865+0000", 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042860570000000002 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733907+0000", 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789999999999998e-05 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.733942+0000", 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5064999999999997e-05 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.224362290999998, 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.651 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:07:50.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:07:50.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:07:50.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:50.656 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.596403710000001, 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:07:50.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.066243243999999, 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:50.663 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:07:50.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:50.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:07:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:07:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:07:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:07:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:50.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:07:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:07:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:07:50.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:07:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:50.671 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:50.671 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:50.674 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:50.675 INFO:teuthology.orchestra.run.smithi079.stdout:2238 trunc from 0xe0e46a4 to 0xccdd37 2024-09-16T14:07:50.675 INFO:teuthology.orchestra.run.smithi079.stdout:2239 trunc from 0xccdd37 to 0x723c234 2024-09-16T14:07:50.675 INFO:teuthology.orchestra.run.smithi079.stdout:2241 read 0x3b9de77 thru 0x3b9ea8c (0xc16 bytes) 2024-09-16T14:07:50.675 INFO:teuthology.orchestra.run.smithi079.stdout:2242 read 0x65dc68f thru 0x65e4eef (0x8861 bytes) 2024-09-16T14:07:50.676 INFO:teuthology.orchestra.run.smithi079.stdout:2243 trunc from 0x723c234 to 0x37b60c 2024-09-16T14:07:50.676 INFO:teuthology.orchestra.run.smithi079.stdout:2246 trunc from 0x37b60c to 0x4fe2a66 2024-09-16T14:07:50.676 INFO:teuthology.orchestra.run.smithi079.stdout:2247 read 0x46f55e4 thru 0x4704201 (0xec1e bytes) 2024-09-16T14:07:50.676 INFO:teuthology.orchestra.run.smithi079.stdout:2248 read 0x1d497e4 thru 0x1d4c751 (0x2f6e bytes) 2024-09-16T14:07:50.676 INFO:teuthology.orchestra.run.smithi079.stdout:2249 trunc from 0x4fe2a66 to 0xd139eec 2024-09-16T14:07:50.677 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:50.665+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:50.677 INFO:teuthology.orchestra.run.smithi079.stdout:2250 trunc from 0xd139eec to 0x4f032fd 2024-09-16T14:07:50.712 INFO:teuthology.orchestra.run.smithi079.stdout:2251 writesame 0x8de623c thru 0x8de6914 data_size 0x1(0x6d9 bytes) 2024-09-16T14:07:50.735 INFO:teuthology.orchestra.run.smithi079.stdout:2252 write 0xc8a5db7 thru 0xc8b0259 (0xa4a3 bytes) 2024-09-16T14:07:50.738 INFO:teuthology.orchestra.run.smithi079.stdout:2253 punch from 0xa89d5fa to 0xa8a08f5, (0x32fb bytes) 2024-09-16T14:07:50.739 INFO:teuthology.orchestra.run.smithi079.stdout:2254 write 0x27d7673 thru 0x27e4f02 (0xd890 bytes) 2024-09-16T14:07:50.742 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.743 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.743 INFO:teuthology.orchestra.run.smithi079.stdout:2258 write 0x24c5214 thru 0x24ca5b6 (0x53a3 bytes) 2024-09-16T14:07:50.747 INFO:teuthology.orchestra.run.smithi079.stdout:2259 write 0x8c2f4b2 thru 0x8c3f113 (0xfc62 bytes) 2024-09-16T14:07:50.751 INFO:teuthology.orchestra.run.smithi079.stdout:2260 read 0x169663 thru 0x17831b (0xecb9 bytes) 2024-09-16T14:07:50.752 INFO:teuthology.orchestra.run.smithi079.stdout:2261 read 0xb02ea1c thru 0xb03935a (0xa93f bytes) 2024-09-16T14:07:50.756 INFO:teuthology.orchestra.run.smithi079.stdout:2262 punch from 0x23e8b17 to 0x23ef2a5, (0x678e bytes) 2024-09-16T14:07:50.756 INFO:teuthology.orchestra.run.smithi079.stdout:2263 read 0x792e6ac thru 0x7935612 (0x6f67 bytes) 2024-09-16T14:07:50.769 INFO:teuthology.orchestra.run.smithi079.stdout:2264 write 0xd9249bf thru 0xd9327ff (0xde41 bytes) 2024-09-16T14:07:50.773 INFO:teuthology.orchestra.run.smithi079.stdout:2267 write 0xb067b7c thru 0xb0735a6 (0xba2b bytes) 2024-09-16T14:07:50.773 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:50.769+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:50.777 INFO:teuthology.orchestra.run.smithi079.stdout:2268 write 0x2f393a7 thru 0x2f3fb94 (0x67ee bytes) 2024-09-16T14:07:50.780 INFO:teuthology.orchestra.run.smithi079.stdout:2269 read 0x342d928 thru 0x34373d9 (0x9ab2 bytes) 2024-09-16T14:07:50.781 INFO:teuthology.orchestra.run.smithi079.stdout:2271 punch from 0x7f5cbe6 to 0x7f601be, (0x35d8 bytes) 2024-09-16T14:07:50.781 INFO:teuthology.orchestra.run.smithi079.stdout:2272 write 0xb71951d thru 0xb71d154 (0x3c38 bytes) 2024-09-16T14:07:50.784 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:50.784 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:50.785 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.785 INFO:teuthology.orchestra.run.smithi079.stdout:2275 read 0x9c84d62 thru 0x9c9065b (0xb8fa bytes) 2024-09-16T14:07:50.785 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.786 INFO:teuthology.orchestra.run.smithi079.stdout:2280 punch from 0xaf3eb91 to 0xaf4bfc9, (0xd438 bytes) 2024-09-16T14:07:50.786 INFO:teuthology.orchestra.run.smithi079.stdout:2281 punch from 0xc10e866 to 0xc10e951, (0xeb bytes) 2024-09-16T14:07:50.786 INFO:teuthology.orchestra.run.smithi079.stdout:2282 read 0x4195e6 thru 0x41fba6 (0x65c1 bytes) 2024-09-16T14:07:50.789 INFO:teuthology.orchestra.run.smithi079.stdout:2283 trunc from 0xd932800 to 0xec53caa 2024-09-16T14:07:50.794 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:50.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:07:50.820 INFO:teuthology.orchestra.run.smithi079.stdout:2284 trunc from 0xec53caa to 0x777010b 2024-09-16T14:07:50.837 INFO:teuthology.orchestra.run.smithi079.stdout:2286 writesame 0x1f00e8e thru 0x1f07877 data_size 0x3(0x69ea bytes) 2024-09-16T14:07:50.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:07:50.867 INFO:teuthology.orchestra.run.smithi079.stdout:2287 write 0xb8a9c65 thru 0xb8b7c3c (0xdfd8 bytes) 2024-09-16T14:07:50.870 INFO:teuthology.orchestra.run.smithi079.stdout:2289 write 0x2e6d2a5 thru 0x2e6ecab (0x1a07 bytes) 2024-09-16T14:07:50.870 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T14:07:50.874 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:50.869+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:50.875 INFO:teuthology.orchestra.run.smithi079.stdout:2290 read 0x3704072 thru 0x3705dd0 (0x1d5f bytes) 2024-09-16T14:07:50.876 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.876 INFO:teuthology.orchestra.run.smithi079.stdout:2294 write 0x3862c9a thru 0x386f65e (0xc9c5 bytes) 2024-09-16T14:07:50.879 INFO:teuthology.orchestra.run.smithi079.stdout:2296 write 0x5a26970 thru 0x5a2edf8 (0x8489 bytes) 2024-09-16T14:07:50.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T14:07:50.891 INFO:teuthology.orchestra.run.smithi079.stdout:2298 write 0xbda639f thru 0xbdaba52 (0x56b4 bytes) 2024-09-16T14:07:50.894 INFO:teuthology.orchestra.run.smithi079.stdout:2299 punch from 0x3f7b474 to 0x3f86287, (0xae13 bytes) 2024-09-16T14:07:50.894 INFO:teuthology.orchestra.run.smithi079.stdout:2300 write 0x1f5c4e7 thru 0x1f644f0 (0x800a bytes) 2024-09-16T14:07:50.899 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:50.899 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:50.899 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:07:50.899 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:07:50.899 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:50.900 INFO:teuthology.orchestra.run.smithi079.stdout:2301 read 0x2f0b63f thru 0x2f102f7 (0x4cb9 bytes) 2024-09-16T14:07:50.901 INFO:teuthology.orchestra.run.smithi079.stdout:2302 write 0x6fbb723 thru 0x6fc9bd1 (0xe4af bytes) 2024-09-16T14:07:50.905 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.905 INFO:teuthology.orchestra.run.smithi079.stdout:2304 read 0x9dbc3f4 thru 0x9dc6def (0xa9fc bytes) 2024-09-16T14:07:50.906 INFO:teuthology.orchestra.run.smithi079.stdout:2305 punch from 0x9cfa254 to 0x9d01629, (0x73d5 bytes) 2024-09-16T14:07:50.906 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.906 INFO:teuthology.orchestra.run.smithi079.stdout:2308 read 0x92495f5 thru 0x92594b1 (0xfebd bytes) 2024-09-16T14:07:50.906 INFO:teuthology.orchestra.run.smithi079.stdout:2309 read 0x26ded02 thru 0x26eea9b (0xfd9a bytes) 2024-09-16T14:07:50.907 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.909 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:50.909 INFO:teuthology.orchestra.run.smithi079.stdout:2311 trunc from 0xbdaba53 to 0xae27dac 2024-09-16T14:07:50.923 INFO:teuthology.orchestra.run.smithi079.stdout:2312 write 0x3a61bc1 thru 0x3a6c4a1 (0xa8e1 bytes) 2024-09-16T14:07:50.927 INFO:teuthology.orchestra.run.smithi079.stdout:2313 trunc from 0xae27dac to 0xc32e767 2024-09-16T14:07:50.943 INFO:teuthology.orchestra.run.smithi079.stdout:2314 write 0xcb8ab8b thru 0xcb92219 (0x768f bytes) 2024-09-16T14:07:50.961 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:50.965 INFO:teuthology.orchestra.run.smithi079.stdout:2315 trunc from 0xcb9221a to 0x1bd9b7f 2024-09-16T14:07:50.975 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:50.969+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:50.980 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:50.998 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:50.999 INFO:teuthology.orchestra.run.smithi079.stdout:2319 write 0x74c916e thru 0x74d79ca (0xe85d bytes) 2024-09-16T14:07:51.012 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9727 3.1a e39/23 3:5b31ba1a:::rbd_data.1136a719415e.000000000000000c:head v 39'121, mlcod=39'118)", 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.791011091000001, 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026296277, 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216593+0000", 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.216595+0000", 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7570000000000001e-06 2024-09-16T14:07:51.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227934+0000", 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011338568 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227959+0000", 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5874e-05 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.227969+0000", 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.397e-06 2024-09-16T14:07:51.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.228059+0000", 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9798999999999999e-05 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.228131+0000", 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2268000000000007e-05 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.242760+0000", 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014629085999999999 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.242890+0000", 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012952800000000001 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12155 3.0 3:06ec4796:::rbd_data.11364aaed44f.0000000000000005:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.477533+0000", 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.530070969000001, 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030057369, 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12155 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.018 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477533+0000", 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477533+0000", 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477536+0000", 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.92e-06 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477551+0000", 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4847e-05 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477586+0000", 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4663999999999999e-05 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477600+0000", 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4297000000000001e-05 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477686+0000", 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6111000000000003e-05 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.477792+0000", 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105863 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.482474+0000", 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046815679999999997 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.507376+0000", 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024902783000000001 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.507397+0000", 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0397000000000001e-05 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.507527+0000", 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000129791 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.507591+0000", 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4128000000000007e-05 2024-09-16T14:07:51.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12183 3.0 3:06ec4796:::rbd_data.11364aaed44f.0000000000000005:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.597948+0000", 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.409656527999999, 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021084459, 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12183 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597948+0000", 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597948+0000", 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597951+0000", 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8490000000000002e-06 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597965+0000", 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4313e-05 2024-09-16T14:07:51.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.597997+0000", 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2224000000000002e-05 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.598008+0000", 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1024e-05 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.598076+0000", 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7966000000000006e-05 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.598158+0000", 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1779000000000004e-05 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.600850+0000", 2024-09-16T14:07:51.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026922679999999998 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.618647+0000", 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017796816999999999 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.618773+0000", 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012578900000000001 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.618926+0000", 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015294199999999999 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.619032+0000", 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106488 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26859 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426144+0000", 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.58146009, 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024786332000000001, 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26859 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426144+0000", 2024-09-16T14:07:51.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426144+0000", 2024-09-16T14:07:51.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426145+0000", 2024-09-16T14:07:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1600000000000001e-07 2024-09-16T14:07:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426148+0000", 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3550000000000001e-06 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426155+0000", 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8800000000000002e-06 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426160+0000", 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8770000000000001e-06 2024-09-16T14:07:51.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.441822+0000", 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015661616 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.441932+0000", 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109998 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.442335+0000", 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040383599999999997 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.447001+0000", 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046651000000000001 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450811+0000", 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038108830000000002 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450861+0000", 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9642999999999999e-05 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450931+0000", 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9528000000000002e-05 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.037 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26860 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426166+0000", 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.581438124, 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024875105000000002, 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26860 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426166+0000", 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426166+0000", 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426167+0000", 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9999999999999997e-07 2024-09-16T14:07:51.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426170+0000", 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4240000000000002e-06 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426177+0000", 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5729999999999996e-06 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426182+0000", 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6480000000000002e-06 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.443169+0000", 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016987825000000002 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.443300+0000", 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013106700000000001 2024-09-16T14:07:51.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.443866+0000", 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056600400000000005 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450801+0000", 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069350749999999997 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450944+0000", 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014229600000000001 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.450985+0000", 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0788000000000003e-05 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451041+0000", 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6805000000000002e-05 2024-09-16T14:07:51.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26861 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426186+0000", 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.581418305, 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025111023, 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26861 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426186+0000", 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426186+0000", 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426187+0000", 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0500000000000003e-07 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426190+0000", 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3579999999999999e-06 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426198+0000", 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7109999999999995e-06 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426202+0000", 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4220000000000002e-06 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.445010+0000", 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018807395000000001 2024-09-16T14:07:51.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.445278+0000", 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026810199999999999 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.445987+0000", 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000709269 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451052+0000", 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050649320000000003 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451138+0000", 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6420000000000003e-05 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451228+0000", 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9659000000000003e-05 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451297+0000", 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.915e-05 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26862 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426207+0000", 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.581397722, 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025300552, 2024-09-16T14:07:51.050 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26862 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426207+0000", 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426207+0000", 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426207+0000", 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0500000000000003e-07 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426211+0000", 2024-09-16T14:07:51.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6330000000000002e-06 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426217+0000", 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5200000000000003e-06 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426222+0000", 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4129999999999999e-06 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.447065+0000", 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020843413000000002 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.447158+0000", 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2799000000000004e-05 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.447745+0000", 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058742899999999999 2024-09-16T14:07:51.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451391+0000", 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003645456 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451403+0000", 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1788e-05 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451446+0000", 2024-09-16T14:07:51.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3677000000000002e-05 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.451507+0000", 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0819000000000002e-05 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26863 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.426227+0000", 2024-09-16T14:07:51.056 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.581377662, 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026712219999999998, 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26863 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426227+0000", 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426227+0000", 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426227+0000", 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8800000000000002e-07 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426231+0000", 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6490000000000001e-06 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426238+0000", 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5780000000000004e-06 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.426242+0000", 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4479999999999996e-06 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.448572+0000", 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022329664999999999 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.448856+0000", 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000284006 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.449757+0000", 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000900897 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.452837+0000", 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030808739999999999 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.452848+0000", 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0807e-05 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.452900+0000", 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1554999999999997e-05 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.452939+0000", 2024-09-16T14:07:51.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9153000000000003e-05 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:26885 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.456520+0000", 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.551084675, 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023841016, 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 26885 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456520+0000", 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456520+0000", 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456520+0000", 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4099999999999998e-07 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456524+0000", 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7639999999999999e-06 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456529+0000", 2024-09-16T14:07:51.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3369999999999999e-06 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.456534+0000", 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2819999999999998e-06 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.474117+0000", 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017583016999999999 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.474214+0000", 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7270999999999998e-05 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.474630+0000", 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041571700000000002 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.476011+0000", 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013808539999999999 2024-09-16T14:07:51.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.480275+0000", 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042646400000000001 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.480323+0000", 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7574000000000002e-05 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.480361+0000", 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7919000000000001e-05 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27014 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.602326+0000", 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.405278574, 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022684165999999999, 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.069 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27014 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602326+0000", 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602326+0000", 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602326+0000", 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602330+0000", 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8539999999999999e-06 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602338+0000", 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4800000000000004e-06 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.602344+0000", 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.618696+0000", 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016351853999999999 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.618751+0000", 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.503e-05 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.618972+0000", 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022084200000000001 2024-09-16T14:07:51.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.621202+0000", 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002230246 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.624900+0000", 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036982180000000001 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.624959+0000", 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9175000000000002e-05 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.625010+0000", 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0575000000000003e-05 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27043 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.075 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.643108+0000", 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.364496203, 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022120593000000001, 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27043 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643108+0000", 2024-09-16T14:07:51.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643108+0000", 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643109+0000", 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.75e-07 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643113+0000", 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0709999999999996e-06 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643122+0000", 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8119999999999997e-06 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643127+0000", 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3800000000000002e-06 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.659795+0000", 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016667996000000001 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.659914+0000", 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118627 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.660734+0000", 2024-09-16T14:07:51.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082016999999999995 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665132+0000", 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043983759999999998 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665141+0000", 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3809999999999999e-06 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665200+0000", 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9382999999999997e-05 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665229+0000", 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8722e-05 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.081 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27044 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.643134+0000", 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.364470682, 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022477451999999998, 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27044 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643134+0000", 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643134+0000", 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643134+0000", 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6199999999999997e-07 2024-09-16T14:07:51.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643146+0000", 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1365999999999999e-05 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643164+0000", 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8479999999999999e-05 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643171+0000", 2024-09-16T14:07:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7769999999999997e-06 2024-09-16T14:07:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.662090+0000", 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018918852999999999 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.662179+0000", 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8844000000000005e-05 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.662638+0000", 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00045959000000000001 2024-09-16T14:07:51.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665246+0000", 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026078239999999999 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665430+0000", 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018435899999999999 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665546+0000", 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011579200000000001 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665611+0000", 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4905000000000003e-05 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.090 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27045 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.643177+0000", 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.364427818999999, 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023067443, 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27045 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643177+0000", 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643177+0000", 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643177+0000", 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2500000000000005e-07 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643181+0000", 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.98e-06 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643191+0000", 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.312e-06 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.643195+0000", 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8860000000000003e-06 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.663428+0000", 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020232624000000001 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.663574+0000", 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014559999999999999 2024-09-16T14:07:51.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.664163+0000", 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058896000000000003 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.665638+0000", 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014754880000000001 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.666019+0000", 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038109299999999999 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.666115+0000", 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6114999999999997e-05 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.666244+0000", 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012866 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.096 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27164 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.779097+0000", 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.228506983999999, 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023825177999999999, 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.097 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27164 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779097+0000", 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779097+0000", 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779098+0000", 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7800000000000002e-07 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779101+0000", 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9280000000000002e-06 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779108+0000", 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3379999999999999e-06 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.779112+0000", 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.442e-06 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.796157+0000", 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017044626 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.796293+0000", 2024-09-16T14:07:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013614000000000001 2024-09-16T14:07:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.796800+0000", 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050668700000000002 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.799036+0000", 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022356569999999998 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.802849+0000", 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038136469999999999 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.802891+0000", 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1646000000000001e-05 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.802923+0000", 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1689000000000001e-05 2024-09-16T14:07:51.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27291 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.913623+0000", 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.093981731, 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022489499999999999, 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.104 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27291 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913623+0000", 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913623+0000", 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913623+0000", 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8200000000000001e-07 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913625+0000", 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913628+0000", 2024-09-16T14:07:51.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.455e-06 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913631+0000", 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5040000000000001e-06 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.931246+0000", 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017615071 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.931309+0000", 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2737999999999995e-05 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.931580+0000", 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027167800000000002 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936026+0000", 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044450779999999999 2024-09-16T14:07:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936032+0000", 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1829999999999999e-06 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936082+0000", 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0331999999999999e-05 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936112+0000", 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0099000000000001e-05 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27292 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.913633+0000", 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.093971164999999, 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023172521000000001, 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27292 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913633+0000", 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913633+0000", 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913634+0000", 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8999999999999998e-07 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913636+0000", 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0829999999999998e-06 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913639+0000", 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.433e-06 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913641+0000", 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.932658+0000", 2024-09-16T14:07:51.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019016043999999999 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.932772+0000", 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011435800000000001 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.933190+0000", 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00041830300000000002 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936125+0000", 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029344639999999999 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936737+0000", 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00061256699999999995 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936782+0000", 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5082e-05 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936806+0000", 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3431e-05 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27293 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.913644+0000", 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.093960294, 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023507556999999998, 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.116 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27293 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913644+0000", 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913644+0000", 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913644+0000", 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8700000000000002e-07 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913647+0000", 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2060000000000001e-06 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913650+0000", 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4149999999999999e-06 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.913653+0000", 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6910000000000002e-06 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.934466+0000", 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020813420999999999 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.934564+0000", 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7666999999999998e-05 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.935003+0000", 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043941999999999999 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.936413+0000", 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014100079999999999 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.937102+0000", 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00068838600000000003 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.937130+0000", 2024-09-16T14:07:51.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8826000000000001e-05 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.937152+0000", 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1229999999999998e-05 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27315 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:39.940451+0000", 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.067153761, 2024-09-16T14:07:51.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023080931999999998, 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27315 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940451+0000", 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940451+0000", 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940451+0000", 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9400000000000001e-07 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940454+0000", 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940457+0000", 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6399999999999999e-06 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.940462+0000", 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1280000000000001e-06 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.957003+0000", 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01654131 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.957112+0000", 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108721 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.957609+0000", 2024-09-16T14:07:51.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000497858 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.958664+0000", 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001054269 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.963434+0000", 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047702530000000003 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.963485+0000", 2024-09-16T14:07:51.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1385999999999999e-05 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:39.963532+0000", 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6257000000000001e-05 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:27372 3.1f 3:fd85bd98:::rbd_object_map.11367eb52311:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-16T14:07:51.128 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:40.007127+0000", 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.000477116000001, 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021560835, 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 27372 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007127+0000", 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007127+0000", 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007128+0000", 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8899999999999998e-07 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007141+0000", 2024-09-16T14:07:51.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3042e-05 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007157+0000", 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6297999999999999e-05 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.007168+0000", 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0475999999999999e-05 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.025119+0000", 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017950679000000001 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.025282+0000", 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016368399999999999 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.025690+0000", 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040757400000000001 2024-09-16T14:07:51.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028269+0000", 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025796209999999998 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028569+0000", 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029910599999999999 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028628+0000", 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9913e-05 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:40.028688+0000", 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9653e-05 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37302 3.1e 3:79d9a7ab:::rbd_data.11361fb616f9.0000000000000005:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.404169+0000", 2024-09-16T14:07:51.134 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.6034349909999999, 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031022287999999999, 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37302 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.404169+0000", 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.404169+0000", 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.404172+0000", 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2239999999999998e-06 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.404183+0000", 2024-09-16T14:07:51.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1866e-05 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.404219+0000", 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5386999999999998e-05 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.404231+0000", 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2527000000000001e-05 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:07:51.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.404321+0000", 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9994000000000006e-05 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.404397+0000", 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5369000000000003e-05 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.410041+0000", 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056446279999999996 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.434991+0000", 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024949854 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.435016+0000", 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4803e-05 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.435109+0000", 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.268e-05 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:07:51.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:07:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.435192+0000", 2024-09-16T14:07:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2955999999999997e-05 2024-09-16T14:07:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:07:51.140 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:07:51.140 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:07:51.142 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:51.143 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:51.134+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:51.143 INFO:teuthology.orchestra.run.smithi079.stdout:2320 write 0x8fa5432 thru 0x8fa8b94 (0x3763 bytes) 2024-09-16T14:07:51.143 INFO:teuthology.orchestra.run.smithi079.stdout:2322 read 0x3dbecfc thru 0x3dc489e (0x5ba3 bytes) 2024-09-16T14:07:51.143 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:51.143 INFO:teuthology.orchestra.run.smithi079.stdout:2325 writesame 0xad99c30 thru 0xada2341 data_size 0x33(0x8712 bytes) 2024-09-16T14:07:51.143 INFO:teuthology.orchestra.run.smithi079.stdout:2326 read 0x69f4a53 thru 0x69fca6d (0x801b bytes) 2024-09-16T14:07:51.143 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:51.143 INFO:teuthology.orchestra.run.smithi079.stdout:2328 read 0x6505a58 thru 0x6514bd4 (0xf17d bytes) 2024-09-16T14:07:51.143 INFO:teuthology.orchestra.run.smithi079.stdout:2332 write 0x7c4016b thru 0x7c46ef1 (0x6d87 bytes) 2024-09-16T14:07:51.144 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:51.144 INFO:teuthology.orchestra.run.smithi079.stdout:2334 trunc from 0xada2342 to 0x5cd89b3 2024-09-16T14:07:51.144 INFO:teuthology.orchestra.run.smithi079.stdout:2335 read 0x4fbcb5 thru 0x509e3d (0xe189 bytes) 2024-09-16T14:07:51.145 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:07:51.145 INFO:teuthology.orchestra.run.smithi079.stdout:2338 write 0x76b2d7e thru 0x76c2b3c (0xfdbf bytes) 2024-09-16T14:07:51.145 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:51.145 INFO:teuthology.orchestra.run.smithi079.stdout:2340 write 0xbc9eefb thru 0xbc9f779 (0x87f bytes) 2024-09-16T14:07:51.146 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:51.146 INFO:teuthology.orchestra.run.smithi079.stdout:2341 trunc from 0xbc9f77a to 0xc7912b 2024-09-16T14:07:51.146 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:51.146 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:51.147 INFO:teuthology.orchestra.run.smithi079.stdout:2343 trunc from 0xc7912b to 0x75e0df5 2024-09-16T14:07:51.154 INFO:teuthology.orchestra.run.smithi079.stdout:2344 writesame 0x5588611 thru 0x559091b data_size 0x1(0x830b bytes) 2024-09-16T14:07:51.156 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:51.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:51.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:51.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:51.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:07:51.158 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:51.159 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:51.159 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:51.159 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:51.180 INFO:teuthology.orchestra.run.smithi079.stdout:2346 write 0x45b7a41 thru 0x45b7f68 (0x528 bytes) 2024-09-16T14:07:51.182 INFO:teuthology.orchestra.run.smithi079.stdout:2348 read 0x2abcc28 thru 0x2abe168 (0x1541 bytes) 2024-09-16T14:07:51.183 INFO:teuthology.orchestra.run.smithi079.stdout:2351 write 0x1da6e0d thru 0x1da8e71 (0x2065 bytes) 2024-09-16T14:07:51.204 INFO:teuthology.orchestra.run.smithi079.stdout:2352 write 0xa43c06e thru 0xa4473cf (0xb362 bytes) 2024-09-16T14:07:51.207 INFO:teuthology.orchestra.run.smithi079.stdout:2353 clone 18 order 18 su 1024 sc 12 2024-09-16T14:07:51.242 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:51.237+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:51.292 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:51.292 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:51.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:51.343 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:51.341+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:51.406 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:51.406 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:51.415 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:51.443 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:51.437+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:51.516 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:51.517 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:51.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:51.571 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:51.566+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:51.627 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:51.627 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:51.636 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:51.645 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:51.641+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:51.733 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:51.733 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:51.742 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:51.745 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:07:51.741+0000 7feff9aa2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22183) UID: 0 2024-09-16T14:07:51.839 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:51.839 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:51.846 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:51.838+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:51.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:51.879 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T14:07:51.879 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:07:51.893 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T14:07:51.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:07:51.947 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:51.942+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:51.960 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:51.961 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:51.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:52.048 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:52.042+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:52.055 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone17 from 0xa4473d0 (overlap 0x37b60c) to 0x216d3a 2024-09-16T14:07:52.066 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:52.066 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:52.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:52.148 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:52.142+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:52.168 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:52.168 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:52.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:52.249 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:52.242+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:52.270 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:52.273 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:07:52.289 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-16T14:07:51.868359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506762328,"num_objects":1474,"num_object_clones":250,"num_object_copies":2948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1474,"num_whiteouts":268,"num_read":62302,"num_read_kb":153719,"num_write":12735,"num_write_kb":102707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9967,"ondisk_log_size":9967,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2492300,"kb_used_data":21900,"kb_used_omap":13,"kb_used_meta":2470258,"kb_avail":752482420,"statfs":{"total":773094113280,"available":770541998080,"internally_reserved":0,"allocated":22425600,"data_stored":111111374,"data_compressed":3480714,"data_compressed_allocated":10362880,"data_compressed_original":104664874,"omap_allocated":13632,"internal_metadata":2529544896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":132508965,"num_objects":122,"num_object_clones":49,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":31,"num_read":11428,"num_read_kb":33534,"num_write":1153,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.043024"},"pg_stats":[{"pgid":"3.1f","version":"56'1148","reported_seq":2482,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.220626+0000","last_change":"2024-09-16T14:07:11.338271+0000","last_active":"2024-09-16T14:07:50.220626+0000","last_peered":"2024-09-16T14:07:50.220626+0000","last_clean":"2024-09-16T14:07:50.220626+0000","last_became_active":"2024-09-16T14:07:09.540900+0000","last_became_peered":"2024-09-16T14:07:09.540900+0000","last_unstale":"2024-09-16T14:07:50.220626+0000","last_undegraded":"2024-09-16T14:07:50.220626+0000","last_fullsized":"2024-09-16T14:07:50.220626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1148,"log_dups_size":0,"ondisk_log_size":1148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:59.820596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233770000000001,"stat_sum":{"num_bytes":14918093,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":6,"num_read":7991,"num_read_kb":9351,"num_write":1999,"num_write_kb":4180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"56'253","reported_seq":992,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.427867+0000","last_change":"2024-09-16T14:07:11.337654+0000","last_active":"2024-09-16T14:07:50.427867+0000","last_peered":"2024-09-16T14:07:50.427867+0000","last_clean":"2024-09-16T14:07:50.427867+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:07:50.427867+0000","last_undegraded":"2024-09-16T14:07:50.427867+0000","last_fullsized":"2024-09-16T14:07:50.427867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060911299999999999,"stat_sum":{"num_bytes":17672323,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":11,"num_read":1174,"num_read_kb":4401,"num_write":277,"num_write_kb":1550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"53'252","reported_seq":3342,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.218111+0000","last_change":"2024-09-16T14:07:11.546356+0000","last_active":"2024-09-16T14:07:50.218111+0000","last_peered":"2024-09-16T14:07:50.218111+0000","last_clean":"2024-09-16T14:07:50.218111+0000","last_became_active":"2024-09-16T14:07:09.552487+0000","last_became_peered":"2024-09-16T14:07:09.552487+0000","last_unstale":"2024-09-16T14:07:50.218111+0000","last_undegraded":"2024-09-16T14:07:50.218111+0000","last_fullsized":"2024-09-16T14:07:50.218111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:53:07.333781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072576699999999999,"stat_sum":{"num_bytes":19702881,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":7,"num_read":4066,"num_read_kb":8924,"num_write":212,"num_write_kb":1553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"57'526","reported_seq":1091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.122627+0000","last_change":"2024-09-16T14:07:11.340416+0000","last_active":"2024-09-16T14:07:51.122627+0000","last_peered":"2024-09-16T14:07:51.122627+0000","last_clean":"2024-09-16T14:07:51.122627+0000","last_became_active":"2024-09-16T14:07:09.551657+0000","last_became_peered":"2024-09-16T14:07:09.551657+0000","last_unstale":"2024-09-16T14:07:51.122627+0000","last_undegraded":"2024-09-16T14:07:51.122627+0000","last_fullsized":"2024-09-16T14:07:51.122627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:49.635426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064309800000000004,"stat_sum":{"num_bytes":8641573,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":6,"num_read":2623,"num_read_kb":3479,"num_write":817,"num_write_kb":1819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"54'275","reported_seq":1521,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.232829+0000","last_change":"2024-09-16T14:07:22.350217+0000","last_active":"2024-09-16T14:07:50.232829+0000","last_peered":"2024-09-16T14:07:50.232829+0000","last_clean":"2024-09-16T14:07:50.232829+0000","last_became_active":"2024-09-16T14:07:22.349546+0000","last_became_peered":"2024-09-16T14:07:22.349546+0000","last_unstale":"2024-09-16T14:07:50.232829+0000","last_undegraded":"2024-09-16T14:07:50.232829+0000","last_fullsized":"2024-09-16T14:07:50.232829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:30.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021275310000000002,"stat_sum":{"num_bytes":11736816,"num_objects":55,"num_object_clones":8,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":11,"num_read":2306,"num_read_kb":3626,"num_write":341,"num_write_kb":2765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"55'296","reported_seq":1744,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.152841+0000","last_change":"2024-09-16T14:07:11.340816+0000","last_active":"2024-09-16T14:07:51.152841+0000","last_peered":"2024-09-16T14:07:51.152841+0000","last_clean":"2024-09-16T14:07:51.152841+0000","last_became_active":"2024-09-16T14:07:09.540237+0000","last_became_peered":"2024-09-16T14:07:09.540237+0000","last_unstale":"2024-09-16T14:07:51.152841+0000","last_undegraded":"2024-09-16T14:07:51.152841+0000","last_fullsized":"2024-09-16T14:07:51.152841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:38.147579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099540499999999999,"stat_sum":{"num_bytes":15304933,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":6,"num_read":2265,"num_read_kb":12374,"num_write":324,"num_write_kb":17514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"56'171","reported_seq":497,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.428489+0000","last_change":"2024-09-16T14:07:22.346713+0000","last_active":"2024-09-16T14:07:50.428489+0000","last_peered":"2024-09-16T14:07:50.428489+0000","last_clean":"2024-09-16T14:07:50.428489+0000","last_became_active":"2024-09-16T14:07:22.346031+0000","last_became_peered":"2024-09-16T14:07:22.346031+0000","last_unstale":"2024-09-16T14:07:50.428489+0000","last_undegraded":"2024-09-16T14:07:50.428489+0000","last_fullsized":"2024-09-16T14:07:50.428489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:39.265638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003350142,"stat_sum":{"num_bytes":9029859,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":7,"num_read":338,"num_read_kb":1068,"num_write":138,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"57'217","reported_seq":754,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.119868+0000","last_change":"2024-09-16T14:07:11.338172+0000","last_active":"2024-09-16T14:07:51.119868+0000","last_peered":"2024-09-16T14:07:51.119868+0000","last_clean":"2024-09-16T14:07:51.119868+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:07:51.119868+0000","last_undegraded":"2024-09-16T14:07:51.119868+0000","last_fullsized":"2024-09-16T14:07:51.119868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148865,"stat_sum":{"num_bytes":22638509,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":8,"num_read":1065,"num_read_kb":3844,"num_write":226,"num_write_kb":3221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"54'695","reported_seq":1007,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.161082+0000","last_change":"2024-09-16T14:07:11.338594+0000","last_active":"2024-09-16T14:07:50.161082+0000","last_peered":"2024-09-16T14:07:50.161082+0000","last_clean":"2024-09-16T14:07:50.161082+0000","last_became_active":"2024-09-16T14:07:09.551606+0000","last_became_peered":"2024-09-16T14:07:09.551606+0000","last_unstale":"2024-09-16T14:07:50.161082+0000","last_undegraded":"2024-09-16T14:07:50.161082+0000","last_fullsized":"2024-09-16T14:07:50.161082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:52.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215788,"stat_sum":{"num_bytes":5335895,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":7,"num_read":3421,"num_read_kb":3616,"num_write":1161,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"55'209","reported_seq":829,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.864704+0000","last_change":"2024-09-16T14:07:11.361262+0000","last_active":"2024-09-16T14:07:49.864704+0000","last_peered":"2024-09-16T14:07:49.864704+0000","last_clean":"2024-09-16T14:07:49.864704+0000","last_became_active":"2024-09-16T14:07:09.554272+0000","last_became_peered":"2024-09-16T14:07:09.554272+0000","last_unstale":"2024-09-16T14:07:49.864704+0000","last_undegraded":"2024-09-16T14:07:49.864704+0000","last_fullsized":"2024-09-16T14:07:49.864704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:10.794154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500454,"stat_sum":{"num_bytes":12182172,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":12,"num_read":898,"num_read_kb":1995,"num_write":192,"num_write_kb":689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"55'164","reported_seq":980,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.850735+0000","last_change":"2024-09-16T14:07:11.343447+0000","last_active":"2024-09-16T14:07:49.850735+0000","last_peered":"2024-09-16T14:07:49.850735+0000","last_clean":"2024-09-16T14:07:49.850735+0000","last_became_active":"2024-09-16T14:07:09.549545+0000","last_became_peered":"2024-09-16T14:07:09.549545+0000","last_unstale":"2024-09-16T14:07:49.850735+0000","last_undegraded":"2024-09-16T14:07:49.850735+0000","last_fullsized":"2024-09-16T14:07:49.850735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:27.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012153120000000001,"stat_sum":{"num_bytes":22744298,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":11,"num_read":838,"num_read_kb":3737,"num_write":209,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"54'228","reported_seq":1422,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.231571+0000","last_change":"2024-09-16T14:07:11.546981+0000","last_active":"2024-09-16T14:07:50.231571+0000","last_peered":"2024-09-16T14:07:50.231571+0000","last_clean":"2024-09-16T14:07:50.231571+0000","last_became_active":"2024-09-16T14:07:09.552582+0000","last_became_peered":"2024-09-16T14:07:09.552582+0000","last_unstale":"2024-09-16T14:07:50.231571+0000","last_undegraded":"2024-09-16T14:07:50.231571+0000","last_fullsized":"2024-09-16T14:07:50.231571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:08.157994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009984429999999999,"stat_sum":{"num_bytes":21414335,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":6,"num_read":2229,"num_read_kb":3807,"num_write":230,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.862203+0000","last_change":"2024-09-16T14:07:02.504261+0000","last_active":"2024-09-16T14:07:49.862203+0000","last_peered":"2024-09-16T14:07:49.862203+0000","last_clean":"2024-09-16T14:07:49.862203+0000","last_became_active":"2024-09-16T14:07:00.475877+0000","last_became_peered":"2024-09-16T14:07:00.475877+0000","last_unstale":"2024-09-16T14:07:49.862203+0000","last_undegraded":"2024-09-16T14:07:49.862203+0000","last_fullsized":"2024-09-16T14:07:49.862203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:15.110271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020129200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"54'352","reported_seq":986,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:07:48.838242+0000","last_change":"2024-09-16T14:07:11.370476+0000","last_active":"2024-09-16T14:07:48.838242+0000","last_peered":"2024-09-16T14:07:48.838242+0000","last_clean":"2024-09-16T14:07:48.838242+0000","last_became_active":"2024-09-16T14:07:09.544524+0000","last_became_peered":"2024-09-16T14:07:09.544524+0000","last_unstale":"2024-09-16T14:07:48.838242+0000","last_undegraded":"2024-09-16T14:07:48.838242+0000","last_fullsized":"2024-09-16T14:07:48.838242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:59.087380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032490359999999999,"stat_sum":{"num_bytes":14400641,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":6,"num_read":1557,"num_read_kb":2097,"num_write":470,"num_write_kb":1271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.850489+0000","last_change":"2024-09-16T14:07:22.350069+0000","last_active":"2024-09-16T14:07:49.850489+0000","last_peered":"2024-09-16T14:07:49.850489+0000","last_clean":"2024-09-16T14:07:49.850489+0000","last_became_active":"2024-09-16T14:07:22.349757+0000","last_became_peered":"2024-09-16T14:07:22.349757+0000","last_unstale":"2024-09-16T14:07:49.850489+0000","last_undegraded":"2024-09-16T14:07:49.850489+0000","last_fullsized":"2024-09-16T14:07:49.850489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:57.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"55'346","reported_seq":2140,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.231144+0000","last_change":"2024-09-16T14:07:22.351951+0000","last_active":"2024-09-16T14:07:50.231144+0000","last_peered":"2024-09-16T14:07:50.231144+0000","last_clean":"2024-09-16T14:07:50.231144+0000","last_became_active":"2024-09-16T14:07:22.351394+0000","last_became_peered":"2024-09-16T14:07:22.351394+0000","last_unstale":"2024-09-16T14:07:50.231144+0000","last_undegraded":"2024-09-16T14:07:50.231144+0000","last_fullsized":"2024-09-16T14:07:50.231144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:55.945287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027484229999999998,"stat_sum":{"num_bytes":25123646,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":7,"num_read":2845,"num_read_kb":20740,"num_write":516,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.862063+0000","last_change":"2024-09-16T14:07:02.502729+0000","last_active":"2024-09-16T14:07:49.862063+0000","last_peered":"2024-09-16T14:07:49.862063+0000","last_clean":"2024-09-16T14:07:49.862063+0000","last_became_active":"2024-09-16T14:07:00.476599+0000","last_became_peered":"2024-09-16T14:07:00.476599+0000","last_unstale":"2024-09-16T14:07:49.862063+0000","last_undegraded":"2024-09-16T14:07:49.862063+0000","last_fullsized":"2024-09-16T14:07:49.862063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:38.332613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056022899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"57'220","reported_seq":920,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.145795+0000","last_change":"2024-09-16T14:07:11.547417+0000","last_active":"2024-09-16T14:07:51.145795+0000","last_peered":"2024-09-16T14:07:51.145795+0000","last_clean":"2024-09-16T14:07:51.145795+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:07:51.145795+0000","last_undegraded":"2024-09-16T14:07:51.145795+0000","last_fullsized":"2024-09-16T14:07:51.145795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097608199999999997,"stat_sum":{"num_bytes":4477957,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":13,"num_read":898,"num_read_kb":2510,"num_write":230,"num_write_kb":1742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.851926+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:07:49.851926+0000","last_peered":"2024-09-16T14:07:49.851926+0000","last_clean":"2024-09-16T14:07:49.851926+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:07:49.851926+0000","last_undegraded":"2024-09-16T14:07:49.851926+0000","last_fullsized":"2024-09-16T14:07:49.851926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"57'387","reported_seq":1093,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.122023+0000","last_change":"2024-09-16T14:07:11.361331+0000","last_active":"2024-09-16T14:07:51.122023+0000","last_peered":"2024-09-16T14:07:51.122023+0000","last_clean":"2024-09-16T14:07:51.122023+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:07:51.122023+0000","last_undegraded":"2024-09-16T14:07:51.122023+0000","last_fullsized":"2024-09-16T14:07:51.122023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062220600000000004,"stat_sum":{"num_bytes":20323895,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":13,"num_read":1692,"num_read_kb":4304,"num_write":513,"num_write_kb":2142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"56'461","reported_seq":2198,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.429894+0000","last_change":"2024-09-16T14:07:11.342816+0000","last_active":"2024-09-16T14:07:50.429894+0000","last_peered":"2024-09-16T14:07:50.429894+0000","last_clean":"2024-09-16T14:07:50.429894+0000","last_became_active":"2024-09-16T14:07:09.549842+0000","last_became_peered":"2024-09-16T14:07:09.549842+0000","last_unstale":"2024-09-16T14:07:50.429894+0000","last_undegraded":"2024-09-16T14:07:50.429894+0000","last_fullsized":"2024-09-16T14:07:50.429894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:47.724999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014467779999999999,"stat_sum":{"num_bytes":17914062,"num_objects":56,"num_object_clones":9,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":13,"num_read":4376,"num_read_kb":4460,"num_write":501,"num_write_kb":3693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":52,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.861830+0000","last_change":"2024-09-16T14:07:02.503836+0000","last_active":"2024-09-16T14:07:49.861830+0000","last_peered":"2024-09-16T14:07:49.861830+0000","last_clean":"2024-09-16T14:07:49.861830+0000","last_became_active":"2024-09-16T14:07:00.572358+0000","last_became_peered":"2024-09-16T14:07:00.572358+0000","last_unstale":"2024-09-16T14:07:49.861830+0000","last_undegraded":"2024-09-16T14:07:49.861830+0000","last_fullsized":"2024-09-16T14:07:49.861830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:28.110370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673887,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"55'626","reported_seq":1750,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.851843+0000","last_change":"2024-09-16T14:07:11.337642+0000","last_active":"2024-09-16T14:07:49.851843+0000","last_peered":"2024-09-16T14:07:49.851843+0000","last_clean":"2024-09-16T14:07:49.851843+0000","last_became_active":"2024-09-16T14:07:09.543015+0000","last_became_peered":"2024-09-16T14:07:09.543015+0000","last_unstale":"2024-09-16T14:07:49.851843+0000","last_undegraded":"2024-09-16T14:07:49.851843+0000","last_fullsized":"2024-09-16T14:07:49.851843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:22:22.438444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076943300000000003,"stat_sum":{"num_bytes":11354552,"num_objects":57,"num_object_clones":12,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":10,"num_read":3535,"num_read_kb":6409,"num_write":1005,"num_write_kb":3126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.951611+0000","last_change":"2024-09-16T14:07:04.121594+0000","last_active":"2024-09-16T14:07:49.951611+0000","last_peered":"2024-09-16T14:07:49.951611+0000","last_clean":"2024-09-16T14:07:49.951611+0000","last_became_active":"2024-09-16T14:07:00.475232+0000","last_became_peered":"2024-09-16T14:07:00.475232+0000","last_unstale":"2024-09-16T14:07:49.951611+0000","last_undegraded":"2024-09-16T14:07:49.951611+0000","last_fullsized":"2024-09-16T14:07:49.951611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.457456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068024199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"57'218","reported_seq":908,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.048203+0000","last_change":"2024-09-16T14:07:11.547270+0000","last_active":"2024-09-16T14:07:51.048203+0000","last_peered":"2024-09-16T14:07:51.048203+0000","last_clean":"2024-09-16T14:07:51.048203+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:07:51.048203+0000","last_undegraded":"2024-09-16T14:07:51.048203+0000","last_fullsized":"2024-09-16T14:07:51.048203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085373199999999995,"stat_sum":{"num_bytes":20970072,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":8,"num_read":1276,"num_read_kb":3568,"num_write":254,"num_write_kb":5169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.861620+0000","last_change":"2024-09-16T14:07:02.502802+0000","last_active":"2024-09-16T14:07:49.861620+0000","last_peered":"2024-09-16T14:07:49.861620+0000","last_clean":"2024-09-16T14:07:49.861620+0000","last_became_active":"2024-09-16T14:07:00.480643+0000","last_became_peered":"2024-09-16T14:07:00.480643+0000","last_unstale":"2024-09-16T14:07:49.861620+0000","last_undegraded":"2024-09-16T14:07:49.861620+0000","last_fullsized":"2024-09-16T14:07:49.861620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:52.536849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"57'221","reported_seq":1059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.121731+0000","last_change":"2024-09-16T14:07:22.350954+0000","last_active":"2024-09-16T14:07:51.121731+0000","last_peered":"2024-09-16T14:07:51.121731+0000","last_clean":"2024-09-16T14:07:51.121731+0000","last_became_active":"2024-09-16T14:07:22.350024+0000","last_became_peered":"2024-09-16T14:07:22.350024+0000","last_unstale":"2024-09-16T14:07:51.121731+0000","last_undegraded":"2024-09-16T14:07:51.121731+0000","last_fullsized":"2024-09-16T14:07:51.121731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:07.634077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002704522,"stat_sum":{"num_bytes":13367215,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":7,"num_read":884,"num_read_kb":2730,"num_write":180,"num_write_kb":781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":51,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.861937+0000","last_change":"2024-09-16T14:07:02.503755+0000","last_active":"2024-09-16T14:07:49.861937+0000","last_peered":"2024-09-16T14:07:49.861937+0000","last_clean":"2024-09-16T14:07:49.861937+0000","last_became_active":"2024-09-16T14:07:00.569748+0000","last_became_peered":"2024-09-16T14:07:00.569748+0000","last_unstale":"2024-09-16T14:07:49.861937+0000","last_undegraded":"2024-09-16T14:07:49.861937+0000","last_fullsized":"2024-09-16T14:07:49.861937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:11.927535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015626559999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":138,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.850768+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:07:49.850768+0000","last_peered":"2024-09-16T14:07:49.850768+0000","last_clean":"2024-09-16T14:07:49.850768+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:07:49.850768+0000","last_undegraded":"2024-09-16T14:07:49.850768+0000","last_fullsized":"2024-09-16T14:07:49.850768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'202","reported_seq":1181,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.951751+0000","last_change":"2024-09-16T14:07:22.352613+0000","last_active":"2024-09-16T14:07:49.951751+0000","last_peered":"2024-09-16T14:07:49.951751+0000","last_clean":"2024-09-16T14:07:49.951751+0000","last_became_active":"2024-09-16T14:07:22.351412+0000","last_became_peered":"2024-09-16T14:07:22.351412+0000","last_unstale":"2024-09-16T14:07:49.951751+0000","last_undegraded":"2024-09-16T14:07:49.951751+0000","last_fullsized":"2024-09-16T14:07:49.951751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:21.471879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022177080000000001,"stat_sum":{"num_bytes":9914785,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":12,"num_read":1082,"num_read_kb":2902,"num_write":214,"num_write_kb":2027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"57'195","reported_seq":1002,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.179384+0000","last_change":"2024-09-16T14:07:11.545923+0000","last_active":"2024-09-16T14:07:51.179384+0000","last_peered":"2024-09-16T14:07:51.179384+0000","last_clean":"2024-09-16T14:07:51.179384+0000","last_became_active":"2024-09-16T14:07:09.552836+0000","last_became_peered":"2024-09-16T14:07:09.552836+0000","last_unstale":"2024-09-16T14:07:51.179384+0000","last_undegraded":"2024-09-16T14:07:51.179384+0000","last_fullsized":"2024-09-16T14:07:51.179384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:41.628255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339700000000001,"stat_sum":{"num_bytes":15652345,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":12,"num_read":863,"num_read_kb":5012,"num_write":198,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"56'327","reported_seq":1345,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.430475+0000","last_change":"2024-09-16T14:07:11.545988+0000","last_active":"2024-09-16T14:07:50.430475+0000","last_peered":"2024-09-16T14:07:50.430475+0000","last_clean":"2024-09-16T14:07:50.430475+0000","last_became_active":"2024-09-16T14:07:09.550304+0000","last_became_peered":"2024-09-16T14:07:09.550304+0000","last_unstale":"2024-09-16T14:07:50.430475+0000","last_undegraded":"2024-09-16T14:07:50.430475+0000","last_fullsized":"2024-09-16T14:07:50.430475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:42.431894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065751299999999998,"stat_sum":{"num_bytes":14541019,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":10,"num_read":1756,"num_read_kb":5254,"num_write":353,"num_write_kb":1116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'192","reported_seq":697,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.207430+0000","last_change":"2024-09-16T14:07:11.377884+0000","last_active":"2024-09-16T14:07:51.207430+0000","last_peered":"2024-09-16T14:07:51.207430+0000","last_clean":"2024-09-16T14:07:51.207430+0000","last_became_active":"2024-09-16T14:07:09.544775+0000","last_became_peered":"2024-09-16T14:07:09.544775+0000","last_unstale":"2024-09-16T14:07:51.207430+0000","last_undegraded":"2024-09-16T14:07:51.207430+0000","last_fullsized":"2024-09-16T14:07:51.207430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:16.336123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056925200000000002,"stat_sum":{"num_bytes":7547686,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":9,"num_read":893,"num_read_kb":1760,"num_write":174,"num_write_kb":593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"57'188","reported_seq":440,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.182208+0000","last_change":"2024-09-16T14:07:11.343448+0000","last_active":"2024-09-16T14:07:51.182208+0000","last_peered":"2024-09-16T14:07:51.182208+0000","last_clean":"2024-09-16T14:07:51.182208+0000","last_became_active":"2024-09-16T14:07:09.553360+0000","last_became_peered":"2024-09-16T14:07:09.553360+0000","last_unstale":"2024-09-16T14:07:51.182208+0000","last_undegraded":"2024-09-16T14:07:51.182208+0000","last_fullsized":"2024-09-16T14:07:51.182208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:28.854037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017881959999999999,"stat_sum":{"num_bytes":19795573,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":12,"num_read":287,"num_read_kb":994,"num_write":166,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"57'126","reported_seq":622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.048837+0000","last_change":"2024-09-16T14:07:11.546511+0000","last_active":"2024-09-16T14:07:51.048837+0000","last_peered":"2024-09-16T14:07:51.048837+0000","last_clean":"2024-09-16T14:07:51.048837+0000","last_became_active":"2024-09-16T14:07:09.559703+0000","last_became_peered":"2024-09-16T14:07:09.559703+0000","last_unstale":"2024-09-16T14:07:51.048837+0000","last_undegraded":"2024-09-16T14:07:51.048837+0000","last_fullsized":"2024-09-16T14:07:51.048837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:06.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074836900000000003,"stat_sum":{"num_bytes":4494935,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":4,"num_read":497,"num_read_kb":788,"num_write":102,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"56'150","reported_seq":690,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.162568+0000","last_change":"2024-09-16T14:07:11.362189+0000","last_active":"2024-09-16T14:07:50.162568+0000","last_peered":"2024-09-16T14:07:50.162568+0000","last_clean":"2024-09-16T14:07:50.162568+0000","last_became_active":"2024-09-16T14:07:09.548973+0000","last_became_peered":"2024-09-16T14:07:09.548973+0000","last_unstale":"2024-09-16T14:07:50.162568+0000","last_undegraded":"2024-09-16T14:07:50.162568+0000","last_fullsized":"2024-09-16T14:07:50.162568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:17.345008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013505660000000001,"stat_sum":{"num_bytes":12107684,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":8,"num_read":1213,"num_read_kb":1029,"num_write":124,"num_write_kb":812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"56'393","reported_seq":2803,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.430158+0000","last_change":"2024-09-16T14:07:11.342735+0000","last_active":"2024-09-16T14:07:50.430158+0000","last_peered":"2024-09-16T14:07:50.430158+0000","last_clean":"2024-09-16T14:07:50.430158+0000","last_became_active":"2024-09-16T14:07:09.559054+0000","last_became_peered":"2024-09-16T14:07:09.559054+0000","last_unstale":"2024-09-16T14:07:50.430158+0000","last_undegraded":"2024-09-16T14:07:50.430158+0000","last_fullsized":"2024-09-16T14:07:50.430158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.926445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372222,"stat_sum":{"num_bytes":35575347,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":4,"num_read":4328,"num_read_kb":9417,"num_write":511,"num_write_kb":5466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"57'175","reported_seq":532,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.121184+0000","last_change":"2024-09-16T14:07:11.340733+0000","last_active":"2024-09-16T14:07:51.121184+0000","last_peered":"2024-09-16T14:07:51.121184+0000","last_clean":"2024-09-16T14:07:51.121184+0000","last_became_active":"2024-09-16T14:07:09.540341+0000","last_became_peered":"2024-09-16T14:07:09.540341+0000","last_unstale":"2024-09-16T14:07:51.121184+0000","last_undegraded":"2024-09-16T14:07:51.121184+0000","last_fullsized":"2024-09-16T14:07:51.121184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:18.953079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012161470000000001,"stat_sum":{"num_bytes":14241201,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":9,"num_read":393,"num_read_kb":4752,"num_write":231,"num_write_kb":3352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"56'302","reported_seq":2231,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.072900+0000","last_change":"2024-09-16T14:07:11.545567+0000","last_active":"2024-09-16T14:07:50.072900+0000","last_peered":"2024-09-16T14:07:50.072900+0000","last_clean":"2024-09-16T14:07:50.072900+0000","last_became_active":"2024-09-16T14:07:09.556146+0000","last_became_peered":"2024-09-16T14:07:09.556146+0000","last_unstale":"2024-09-16T14:07:50.072900+0000","last_undegraded":"2024-09-16T14:07:50.072900+0000","last_fullsized":"2024-09-16T14:07:50.072900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:29.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037127800000000001,"stat_sum":{"num_bytes":14177913,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":2,"num_read":2661,"num_read_kb":7012,"num_write":384,"num_write_kb":5877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"56'208","reported_seq":874,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.425618+0000","last_change":"2024-09-16T14:07:11.547712+0000","last_active":"2024-09-16T14:07:50.425618+0000","last_peered":"2024-09-16T14:07:50.425618+0000","last_clean":"2024-09-16T14:07:50.425618+0000","last_became_active":"2024-09-16T14:07:09.553092+0000","last_became_peered":"2024-09-16T14:07:09.553092+0000","last_unstale":"2024-09-16T14:07:50.425618+0000","last_undegraded":"2024-09-16T14:07:50.425618+0000","last_fullsized":"2024-09-16T14:07:50.425618+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:54.650445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012662420000000001,"stat_sum":{"num_bytes":18867750,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":4,"num_read":1178,"num_read_kb":1931,"num_write":184,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"57'169","reported_seq":947,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.122837+0000","last_change":"2024-09-16T14:07:11.546794+0000","last_active":"2024-09-16T14:07:51.122837+0000","last_peered":"2024-09-16T14:07:51.122837+0000","last_clean":"2024-09-16T14:07:51.122837+0000","last_became_active":"2024-09-16T14:07:09.555363+0000","last_became_peered":"2024-09-16T14:07:09.555363+0000","last_unstale":"2024-09-16T14:07:51.122837+0000","last_undegraded":"2024-09-16T14:07:51.122837+0000","last_fullsized":"2024-09-16T14:07:51.122837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:05.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066482800000000001,"stat_sum":{"num_bytes":30001976,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":7,"num_read":808,"num_read_kb":5773,"num_write":196,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":506171941,"num_objects":1470,"num_object_clones":250,"num_object_copies":2940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1470,"num_whiteouts":268,"num_read":62236,"num_read_kb":153664,"num_write":12632,"num_write_kb":101077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17162240,"data_stored":107730170,"data_compressed":3500720,"data_compressed_allocated":10326016,"data_compressed_original":103575338,"omap_allocated":0,"internal_metadata":0},"log_size":9892,"ondisk_log_size":9892,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307312,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064528,"statfs":{"total":96636764160,"available":96322076672,"internally_reserved":0,"allocated":1667072,"data_stored":4479172,"data_compressed":174898,"data_compressed_allocated":438272,"data_compressed_original":3837952,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]}]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308880,"kb_used_data":3208,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062960,"statfs":{"total":96636764160,"available":96320471040,"internally_reserved":0,"allocated":3284992,"data_stored":20149602,"data_compressed":508079,"data_compressed_allocated":1765376,"data_compressed_original":19380694,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317184,"kb_used_data":3188,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94054656,"statfs":{"total":96636764160,"available":96311967744,"internally_reserved":0,"allocated":3264512,"data_stored":20382793,"data_compressed":479695,"data_compressed_allocated":1703936,"data_compressed_original":19517190,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312696,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059144,"statfs":{"total":96636764160,"available":96316563456,"internally_reserved":0,"allocated":2932736,"data_stored":12159821,"data_compressed":563604,"data_compressed_allocated":1290240,"data_compressed_original":11292164,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312316,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059524,"statfs":{"total":96636764160,"available":96316952576,"internally_reserved":0,"allocated":2539520,"data_stored":7537972,"data_compressed":347665,"data_compressed_allocated":921600,"data_compressed_original":6684349,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312664,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059176,"statfs":{"total":96636764160,"available":96316596224,"internally_reserved":0,"allocated":2891776,"data_stored":16464202,"data_compressed":465149,"data_compressed_allocated":1482752,"data_compressed_original":15699968,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312960,"kb_used_data":3116,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058880,"statfs":{"total":96636764160,"available":96316293120,"internally_reserved":0,"allocated":3190784,"data_stored":14030516,"data_compressed":585319,"data_compressed_allocated":1445888,"data_compressed_original":13057471,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308288,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063552,"statfs":{"total":96636764160,"available":96321077248,"internally_reserved":0,"allocated":2654208,"data_stored":15907296,"data_compressed":356305,"data_compressed_allocated":1314816,"data_compressed_original":15195086,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2539520,"data_stored":13746461,"data_compressed":585319,"data_compressed_allocated":1445888,"data_compressed_original":13057471,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2183168,"data_stored":15589779,"data_compressed":459758,"data_compressed_allocated":1445888,"data_compressed_original":15126528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":7253917,"data_compressed":347665,"data_compressed_allocated":921600,"data_compressed_original":6684349,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2277376,"data_stored":11875747,"data_compressed":563604,"data_compressed_allocated":1290240,"data_compressed_original":11292164,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":15704771,"data_compressed":387095,"data_compressed_allocated":1351680,"data_compressed_original":15264718,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2555904,"data_stored":19508370,"data_compressed":474304,"data_compressed_allocated":1667072,"data_compressed_original":18943750,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":19865528,"data_compressed":508079,"data_compressed_allocated":1765376,"data_compressed_original":19380694,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1015808,"data_stored":4185597,"data_compressed":174896,"data_compressed_allocated":438272,"data_compressed_original":3825664,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:07:52.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:07:52.302 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-16T14:07:52.308 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:52.308 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:52.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:52.323 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:52.330 INFO:teuthology.orchestra.run.smithi079.stdout:2358 trunc from 0xa4473d0 to 0x630f23b 2024-09-16T14:07:52.349 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:52.341+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:52.456 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:52.445+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:52.471 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:52.471 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:52.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:52.552 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:52.545+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:52.603 INFO:teuthology.orchestra.run.smithi079.stdout:2359 write 0xce53125 thru 0xce5676e (0x364a bytes) 2024-09-16T14:07:52.636 INFO:teuthology.orchestra.run.smithi079.stdout:2360 writesame 0x260005c thru 0x2607dde data_size 0x17(0x7d83 bytes) 2024-09-16T14:07:52.653 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:52.646+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:52.675 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:52.675 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:52.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:52.754 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:52.749+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:52.781 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:07:52.802 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:07:52.815 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-16T14:07:51.868359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506762328,"num_objects":1474,"num_object_clones":250,"num_object_copies":2948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1474,"num_whiteouts":268,"num_read":62302,"num_read_kb":153719,"num_write":12735,"num_write_kb":102707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9967,"ondisk_log_size":9967,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2492300,"kb_used_data":21900,"kb_used_omap":13,"kb_used_meta":2470258,"kb_avail":752482420,"statfs":{"total":773094113280,"available":770541998080,"internally_reserved":0,"allocated":22425600,"data_stored":111111374,"data_compressed":3480714,"data_compressed_allocated":10362880,"data_compressed_original":104664874,"omap_allocated":13632,"internal_metadata":2529544896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":132508965,"num_objects":122,"num_object_clones":49,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":31,"num_read":11428,"num_read_kb":33534,"num_write":1153,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.043024"},"pg_stats":[{"pgid":"3.1f","version":"56'1148","reported_seq":2482,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.220626+0000","last_change":"2024-09-16T14:07:11.338271+0000","last_active":"2024-09-16T14:07:50.220626+0000","last_peered":"2024-09-16T14:07:50.220626+0000","last_clean":"2024-09-16T14:07:50.220626+0000","last_became_active":"2024-09-16T14:07:09.540900+0000","last_became_peered":"2024-09-16T14:07:09.540900+0000","last_unstale":"2024-09-16T14:07:50.220626+0000","last_undegraded":"2024-09-16T14:07:50.220626+0000","last_fullsized":"2024-09-16T14:07:50.220626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1148,"log_dups_size":0,"ondisk_log_size":1148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:59.820596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233770000000001,"stat_sum":{"num_bytes":14918093,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":6,"num_read":7991,"num_read_kb":9351,"num_write":1999,"num_write_kb":4180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"56'253","reported_seq":992,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.427867+0000","last_change":"2024-09-16T14:07:11.337654+0000","last_active":"2024-09-16T14:07:50.427867+0000","last_peered":"2024-09-16T14:07:50.427867+0000","last_clean":"2024-09-16T14:07:50.427867+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:07:50.427867+0000","last_undegraded":"2024-09-16T14:07:50.427867+0000","last_fullsized":"2024-09-16T14:07:50.427867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060911299999999999,"stat_sum":{"num_bytes":17672323,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":11,"num_read":1174,"num_read_kb":4401,"num_write":277,"num_write_kb":1550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"53'252","reported_seq":3342,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.218111+0000","last_change":"2024-09-16T14:07:11.546356+0000","last_active":"2024-09-16T14:07:50.218111+0000","last_peered":"2024-09-16T14:07:50.218111+0000","last_clean":"2024-09-16T14:07:50.218111+0000","last_became_active":"2024-09-16T14:07:09.552487+0000","last_became_peered":"2024-09-16T14:07:09.552487+0000","last_unstale":"2024-09-16T14:07:50.218111+0000","last_undegraded":"2024-09-16T14:07:50.218111+0000","last_fullsized":"2024-09-16T14:07:50.218111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:53:07.333781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072576699999999999,"stat_sum":{"num_bytes":19702881,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":7,"num_read":4066,"num_read_kb":8924,"num_write":212,"num_write_kb":1553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"57'526","reported_seq":1091,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.122627+0000","last_change":"2024-09-16T14:07:11.340416+0000","last_active":"2024-09-16T14:07:51.122627+0000","last_peered":"2024-09-16T14:07:51.122627+0000","last_clean":"2024-09-16T14:07:51.122627+0000","last_became_active":"2024-09-16T14:07:09.551657+0000","last_became_peered":"2024-09-16T14:07:09.551657+0000","last_unstale":"2024-09-16T14:07:51.122627+0000","last_undegraded":"2024-09-16T14:07:51.122627+0000","last_fullsized":"2024-09-16T14:07:51.122627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:49.635426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064309800000000004,"stat_sum":{"num_bytes":8641573,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":6,"num_read":2623,"num_read_kb":3479,"num_write":817,"num_write_kb":1819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"54'275","reported_seq":1521,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.232829+0000","last_change":"2024-09-16T14:07:22.350217+0000","last_active":"2024-09-16T14:07:50.232829+0000","last_peered":"2024-09-16T14:07:50.232829+0000","last_clean":"2024-09-16T14:07:50.232829+0000","last_became_active":"2024-09-16T14:07:22.349546+0000","last_became_peered":"2024-09-16T14:07:22.349546+0000","last_unstale":"2024-09-16T14:07:50.232829+0000","last_undegraded":"2024-09-16T14:07:50.232829+0000","last_fullsized":"2024-09-16T14:07:50.232829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:30.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021275310000000002,"stat_sum":{"num_bytes":11736816,"num_objects":55,"num_object_clones":8,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":11,"num_read":2306,"num_read_kb":3626,"num_write":341,"num_write_kb":2765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"55'296","reported_seq":1744,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.152841+0000","last_change":"2024-09-16T14:07:11.340816+0000","last_active":"2024-09-16T14:07:51.152841+0000","last_peered":"2024-09-16T14:07:51.152841+0000","last_clean":"2024-09-16T14:07:51.152841+0000","last_became_active":"2024-09-16T14:07:09.540237+0000","last_became_peered":"2024-09-16T14:07:09.540237+0000","last_unstale":"2024-09-16T14:07:51.152841+0000","last_undegraded":"2024-09-16T14:07:51.152841+0000","last_fullsized":"2024-09-16T14:07:51.152841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:38.147579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099540499999999999,"stat_sum":{"num_bytes":15304933,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":6,"num_read":2265,"num_read_kb":12374,"num_write":324,"num_write_kb":17514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"56'171","reported_seq":497,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.428489+0000","last_change":"2024-09-16T14:07:22.346713+0000","last_active":"2024-09-16T14:07:50.428489+0000","last_peered":"2024-09-16T14:07:50.428489+0000","last_clean":"2024-09-16T14:07:50.428489+0000","last_became_active":"2024-09-16T14:07:22.346031+0000","last_became_peered":"2024-09-16T14:07:22.346031+0000","last_unstale":"2024-09-16T14:07:50.428489+0000","last_undegraded":"2024-09-16T14:07:50.428489+0000","last_fullsized":"2024-09-16T14:07:50.428489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:39.265638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003350142,"stat_sum":{"num_bytes":9029859,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":7,"num_read":338,"num_read_kb":1068,"num_write":138,"num_write_kb":458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"57'217","reported_seq":754,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.119868+0000","last_change":"2024-09-16T14:07:11.338172+0000","last_active":"2024-09-16T14:07:51.119868+0000","last_peered":"2024-09-16T14:07:51.119868+0000","last_clean":"2024-09-16T14:07:51.119868+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:07:51.119868+0000","last_undegraded":"2024-09-16T14:07:51.119868+0000","last_fullsized":"2024-09-16T14:07:51.119868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148865,"stat_sum":{"num_bytes":22638509,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":8,"num_read":1065,"num_read_kb":3844,"num_write":226,"num_write_kb":3221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"54'695","reported_seq":1007,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.161082+0000","last_change":"2024-09-16T14:07:11.338594+0000","last_active":"2024-09-16T14:07:50.161082+0000","last_peered":"2024-09-16T14:07:50.161082+0000","last_clean":"2024-09-16T14:07:50.161082+0000","last_became_active":"2024-09-16T14:07:09.551606+0000","last_became_peered":"2024-09-16T14:07:09.551606+0000","last_unstale":"2024-09-16T14:07:50.161082+0000","last_undegraded":"2024-09-16T14:07:50.161082+0000","last_fullsized":"2024-09-16T14:07:50.161082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:52.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215788,"stat_sum":{"num_bytes":5335895,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":7,"num_read":3421,"num_read_kb":3616,"num_write":1161,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"55'209","reported_seq":829,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.864704+0000","last_change":"2024-09-16T14:07:11.361262+0000","last_active":"2024-09-16T14:07:49.864704+0000","last_peered":"2024-09-16T14:07:49.864704+0000","last_clean":"2024-09-16T14:07:49.864704+0000","last_became_active":"2024-09-16T14:07:09.554272+0000","last_became_peered":"2024-09-16T14:07:09.554272+0000","last_unstale":"2024-09-16T14:07:49.864704+0000","last_undegraded":"2024-09-16T14:07:49.864704+0000","last_fullsized":"2024-09-16T14:07:49.864704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:10.794154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500454,"stat_sum":{"num_bytes":12182172,"num_objects":48,"num_object_clones":8,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":12,"num_read":898,"num_read_kb":1995,"num_write":192,"num_write_kb":689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"55'164","reported_seq":980,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.850735+0000","last_change":"2024-09-16T14:07:11.343447+0000","last_active":"2024-09-16T14:07:49.850735+0000","last_peered":"2024-09-16T14:07:49.850735+0000","last_clean":"2024-09-16T14:07:49.850735+0000","last_became_active":"2024-09-16T14:07:09.549545+0000","last_became_peered":"2024-09-16T14:07:09.549545+0000","last_unstale":"2024-09-16T14:07:49.850735+0000","last_undegraded":"2024-09-16T14:07:49.850735+0000","last_fullsized":"2024-09-16T14:07:49.850735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:27.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012153120000000001,"stat_sum":{"num_bytes":22744298,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":11,"num_read":838,"num_read_kb":3737,"num_write":209,"num_write_kb":3645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"54'228","reported_seq":1422,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.231571+0000","last_change":"2024-09-16T14:07:11.546981+0000","last_active":"2024-09-16T14:07:50.231571+0000","last_peered":"2024-09-16T14:07:50.231571+0000","last_clean":"2024-09-16T14:07:50.231571+0000","last_became_active":"2024-09-16T14:07:09.552582+0000","last_became_peered":"2024-09-16T14:07:09.552582+0000","last_unstale":"2024-09-16T14:07:50.231571+0000","last_undegraded":"2024-09-16T14:07:50.231571+0000","last_fullsized":"2024-09-16T14:07:50.231571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:08.157994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009984429999999999,"stat_sum":{"num_bytes":21414335,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":6,"num_read":2229,"num_read_kb":3807,"num_write":230,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.862203+0000","last_change":"2024-09-16T14:07:02.504261+0000","last_active":"2024-09-16T14:07:49.862203+0000","last_peered":"2024-09-16T14:07:49.862203+0000","last_clean":"2024-09-16T14:07:49.862203+0000","last_became_active":"2024-09-16T14:07:00.475877+0000","last_became_peered":"2024-09-16T14:07:00.475877+0000","last_unstale":"2024-09-16T14:07:49.862203+0000","last_undegraded":"2024-09-16T14:07:49.862203+0000","last_fullsized":"2024-09-16T14:07:49.862203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:15.110271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020129200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"54'352","reported_seq":986,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:07:48.838242+0000","last_change":"2024-09-16T14:07:11.370476+0000","last_active":"2024-09-16T14:07:48.838242+0000","last_peered":"2024-09-16T14:07:48.838242+0000","last_clean":"2024-09-16T14:07:48.838242+0000","last_became_active":"2024-09-16T14:07:09.544524+0000","last_became_peered":"2024-09-16T14:07:09.544524+0000","last_unstale":"2024-09-16T14:07:48.838242+0000","last_undegraded":"2024-09-16T14:07:48.838242+0000","last_fullsized":"2024-09-16T14:07:48.838242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:59.087380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032490359999999999,"stat_sum":{"num_bytes":14400641,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":6,"num_read":1557,"num_read_kb":2097,"num_write":470,"num_write_kb":1271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":34,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.850489+0000","last_change":"2024-09-16T14:07:22.350069+0000","last_active":"2024-09-16T14:07:49.850489+0000","last_peered":"2024-09-16T14:07:49.850489+0000","last_clean":"2024-09-16T14:07:49.850489+0000","last_became_active":"2024-09-16T14:07:22.349757+0000","last_became_peered":"2024-09-16T14:07:22.349757+0000","last_unstale":"2024-09-16T14:07:49.850489+0000","last_undegraded":"2024-09-16T14:07:49.850489+0000","last_fullsized":"2024-09-16T14:07:49.850489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:57.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"55'346","reported_seq":2140,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.231144+0000","last_change":"2024-09-16T14:07:22.351951+0000","last_active":"2024-09-16T14:07:50.231144+0000","last_peered":"2024-09-16T14:07:50.231144+0000","last_clean":"2024-09-16T14:07:50.231144+0000","last_became_active":"2024-09-16T14:07:22.351394+0000","last_became_peered":"2024-09-16T14:07:22.351394+0000","last_unstale":"2024-09-16T14:07:50.231144+0000","last_undegraded":"2024-09-16T14:07:50.231144+0000","last_fullsized":"2024-09-16T14:07:50.231144+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:55.945287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027484229999999998,"stat_sum":{"num_bytes":25123646,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":7,"num_read":2845,"num_read_kb":20740,"num_write":516,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.862063+0000","last_change":"2024-09-16T14:07:02.502729+0000","last_active":"2024-09-16T14:07:49.862063+0000","last_peered":"2024-09-16T14:07:49.862063+0000","last_clean":"2024-09-16T14:07:49.862063+0000","last_became_active":"2024-09-16T14:07:00.476599+0000","last_became_peered":"2024-09-16T14:07:00.476599+0000","last_unstale":"2024-09-16T14:07:49.862063+0000","last_undegraded":"2024-09-16T14:07:49.862063+0000","last_fullsized":"2024-09-16T14:07:49.862063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:38.332613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056022899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"57'220","reported_seq":920,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.145795+0000","last_change":"2024-09-16T14:07:11.547417+0000","last_active":"2024-09-16T14:07:51.145795+0000","last_peered":"2024-09-16T14:07:51.145795+0000","last_clean":"2024-09-16T14:07:51.145795+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:07:51.145795+0000","last_undegraded":"2024-09-16T14:07:51.145795+0000","last_fullsized":"2024-09-16T14:07:51.145795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097608199999999997,"stat_sum":{"num_bytes":4477957,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":13,"num_read":898,"num_read_kb":2510,"num_write":230,"num_write_kb":1742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.851926+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:07:49.851926+0000","last_peered":"2024-09-16T14:07:49.851926+0000","last_clean":"2024-09-16T14:07:49.851926+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:07:49.851926+0000","last_undegraded":"2024-09-16T14:07:49.851926+0000","last_fullsized":"2024-09-16T14:07:49.851926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"57'387","reported_seq":1093,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.122023+0000","last_change":"2024-09-16T14:07:11.361331+0000","last_active":"2024-09-16T14:07:51.122023+0000","last_peered":"2024-09-16T14:07:51.122023+0000","last_clean":"2024-09-16T14:07:51.122023+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:07:51.122023+0000","last_undegraded":"2024-09-16T14:07:51.122023+0000","last_fullsized":"2024-09-16T14:07:51.122023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062220600000000004,"stat_sum":{"num_bytes":20323895,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":13,"num_read":1692,"num_read_kb":4304,"num_write":513,"num_write_kb":2142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"56'461","reported_seq":2198,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.429894+0000","last_change":"2024-09-16T14:07:11.342816+0000","last_active":"2024-09-16T14:07:50.429894+0000","last_peered":"2024-09-16T14:07:50.429894+0000","last_clean":"2024-09-16T14:07:50.429894+0000","last_became_active":"2024-09-16T14:07:09.549842+0000","last_became_peered":"2024-09-16T14:07:09.549842+0000","last_unstale":"2024-09-16T14:07:50.429894+0000","last_undegraded":"2024-09-16T14:07:50.429894+0000","last_fullsized":"2024-09-16T14:07:50.429894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:47.724999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014467779999999999,"stat_sum":{"num_bytes":17914062,"num_objects":56,"num_object_clones":9,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":13,"num_read":4376,"num_read_kb":4460,"num_write":501,"num_write_kb":3693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":52,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.861830+0000","last_change":"2024-09-16T14:07:02.503836+0000","last_active":"2024-09-16T14:07:49.861830+0000","last_peered":"2024-09-16T14:07:49.861830+0000","last_clean":"2024-09-16T14:07:49.861830+0000","last_became_active":"2024-09-16T14:07:00.572358+0000","last_became_peered":"2024-09-16T14:07:00.572358+0000","last_unstale":"2024-09-16T14:07:49.861830+0000","last_undegraded":"2024-09-16T14:07:49.861830+0000","last_fullsized":"2024-09-16T14:07:49.861830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:28.110370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673887,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"55'626","reported_seq":1750,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.851843+0000","last_change":"2024-09-16T14:07:11.337642+0000","last_active":"2024-09-16T14:07:49.851843+0000","last_peered":"2024-09-16T14:07:49.851843+0000","last_clean":"2024-09-16T14:07:49.851843+0000","last_became_active":"2024-09-16T14:07:09.543015+0000","last_became_peered":"2024-09-16T14:07:09.543015+0000","last_unstale":"2024-09-16T14:07:49.851843+0000","last_undegraded":"2024-09-16T14:07:49.851843+0000","last_fullsized":"2024-09-16T14:07:49.851843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:22:22.438444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076943300000000003,"stat_sum":{"num_bytes":11354552,"num_objects":57,"num_object_clones":12,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":10,"num_read":3535,"num_read_kb":6409,"num_write":1005,"num_write_kb":3126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.951611+0000","last_change":"2024-09-16T14:07:04.121594+0000","last_active":"2024-09-16T14:07:49.951611+0000","last_peered":"2024-09-16T14:07:49.951611+0000","last_clean":"2024-09-16T14:07:49.951611+0000","last_became_active":"2024-09-16T14:07:00.475232+0000","last_became_peered":"2024-09-16T14:07:00.475232+0000","last_unstale":"2024-09-16T14:07:49.951611+0000","last_undegraded":"2024-09-16T14:07:49.951611+0000","last_fullsized":"2024-09-16T14:07:49.951611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.457456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068024199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"57'218","reported_seq":908,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.048203+0000","last_change":"2024-09-16T14:07:11.547270+0000","last_active":"2024-09-16T14:07:51.048203+0000","last_peered":"2024-09-16T14:07:51.048203+0000","last_clean":"2024-09-16T14:07:51.048203+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:07:51.048203+0000","last_undegraded":"2024-09-16T14:07:51.048203+0000","last_fullsized":"2024-09-16T14:07:51.048203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085373199999999995,"stat_sum":{"num_bytes":20970072,"num_objects":46,"num_object_clones":7,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":8,"num_read":1276,"num_read_kb":3568,"num_write":254,"num_write_kb":5169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.861620+0000","last_change":"2024-09-16T14:07:02.502802+0000","last_active":"2024-09-16T14:07:49.861620+0000","last_peered":"2024-09-16T14:07:49.861620+0000","last_clean":"2024-09-16T14:07:49.861620+0000","last_became_active":"2024-09-16T14:07:00.480643+0000","last_became_peered":"2024-09-16T14:07:00.480643+0000","last_unstale":"2024-09-16T14:07:49.861620+0000","last_undegraded":"2024-09-16T14:07:49.861620+0000","last_fullsized":"2024-09-16T14:07:49.861620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:52.536849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"57'221","reported_seq":1059,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.121731+0000","last_change":"2024-09-16T14:07:22.350954+0000","last_active":"2024-09-16T14:07:51.121731+0000","last_peered":"2024-09-16T14:07:51.121731+0000","last_clean":"2024-09-16T14:07:51.121731+0000","last_became_active":"2024-09-16T14:07:22.350024+0000","last_became_peered":"2024-09-16T14:07:22.350024+0000","last_unstale":"2024-09-16T14:07:51.121731+0000","last_undegraded":"2024-09-16T14:07:51.121731+0000","last_fullsized":"2024-09-16T14:07:51.121731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:07.634077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002704522,"stat_sum":{"num_bytes":13367215,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":7,"num_read":884,"num_read_kb":2730,"num_write":180,"num_write_kb":781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":51,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.861937+0000","last_change":"2024-09-16T14:07:02.503755+0000","last_active":"2024-09-16T14:07:49.861937+0000","last_peered":"2024-09-16T14:07:49.861937+0000","last_clean":"2024-09-16T14:07:49.861937+0000","last_became_active":"2024-09-16T14:07:00.569748+0000","last_became_peered":"2024-09-16T14:07:00.569748+0000","last_unstale":"2024-09-16T14:07:49.861937+0000","last_undegraded":"2024-09-16T14:07:49.861937+0000","last_fullsized":"2024-09-16T14:07:49.861937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:11.927535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015626559999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":138,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.850768+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:07:49.850768+0000","last_peered":"2024-09-16T14:07:49.850768+0000","last_clean":"2024-09-16T14:07:49.850768+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:07:49.850768+0000","last_undegraded":"2024-09-16T14:07:49.850768+0000","last_fullsized":"2024-09-16T14:07:49.850768+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'202","reported_seq":1181,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:49.951751+0000","last_change":"2024-09-16T14:07:22.352613+0000","last_active":"2024-09-16T14:07:49.951751+0000","last_peered":"2024-09-16T14:07:49.951751+0000","last_clean":"2024-09-16T14:07:49.951751+0000","last_became_active":"2024-09-16T14:07:22.351412+0000","last_became_peered":"2024-09-16T14:07:22.351412+0000","last_unstale":"2024-09-16T14:07:49.951751+0000","last_undegraded":"2024-09-16T14:07:49.951751+0000","last_fullsized":"2024-09-16T14:07:49.951751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:21.471879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022177080000000001,"stat_sum":{"num_bytes":9914785,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":12,"num_read":1082,"num_read_kb":2902,"num_write":214,"num_write_kb":2027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"57'195","reported_seq":1002,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.179384+0000","last_change":"2024-09-16T14:07:11.545923+0000","last_active":"2024-09-16T14:07:51.179384+0000","last_peered":"2024-09-16T14:07:51.179384+0000","last_clean":"2024-09-16T14:07:51.179384+0000","last_became_active":"2024-09-16T14:07:09.552836+0000","last_became_peered":"2024-09-16T14:07:09.552836+0000","last_unstale":"2024-09-16T14:07:51.179384+0000","last_undegraded":"2024-09-16T14:07:51.179384+0000","last_fullsized":"2024-09-16T14:07:51.179384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:41.628255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339700000000001,"stat_sum":{"num_bytes":15652345,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":12,"num_read":863,"num_read_kb":5012,"num_write":198,"num_write_kb":3088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"56'327","reported_seq":1345,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.430475+0000","last_change":"2024-09-16T14:07:11.545988+0000","last_active":"2024-09-16T14:07:50.430475+0000","last_peered":"2024-09-16T14:07:50.430475+0000","last_clean":"2024-09-16T14:07:50.430475+0000","last_became_active":"2024-09-16T14:07:09.550304+0000","last_became_peered":"2024-09-16T14:07:09.550304+0000","last_unstale":"2024-09-16T14:07:50.430475+0000","last_undegraded":"2024-09-16T14:07:50.430475+0000","last_fullsized":"2024-09-16T14:07:50.430475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:42.431894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065751299999999998,"stat_sum":{"num_bytes":14541019,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":10,"num_read":1756,"num_read_kb":5254,"num_write":353,"num_write_kb":1116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'192","reported_seq":697,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.207430+0000","last_change":"2024-09-16T14:07:11.377884+0000","last_active":"2024-09-16T14:07:51.207430+0000","last_peered":"2024-09-16T14:07:51.207430+0000","last_clean":"2024-09-16T14:07:51.207430+0000","last_became_active":"2024-09-16T14:07:09.544775+0000","last_became_peered":"2024-09-16T14:07:09.544775+0000","last_unstale":"2024-09-16T14:07:51.207430+0000","last_undegraded":"2024-09-16T14:07:51.207430+0000","last_fullsized":"2024-09-16T14:07:51.207430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:16.336123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056925200000000002,"stat_sum":{"num_bytes":7547686,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":9,"num_read":893,"num_read_kb":1760,"num_write":174,"num_write_kb":593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"57'188","reported_seq":440,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.182208+0000","last_change":"2024-09-16T14:07:11.343448+0000","last_active":"2024-09-16T14:07:51.182208+0000","last_peered":"2024-09-16T14:07:51.182208+0000","last_clean":"2024-09-16T14:07:51.182208+0000","last_became_active":"2024-09-16T14:07:09.553360+0000","last_became_peered":"2024-09-16T14:07:09.553360+0000","last_unstale":"2024-09-16T14:07:51.182208+0000","last_undegraded":"2024-09-16T14:07:51.182208+0000","last_fullsized":"2024-09-16T14:07:51.182208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:28.854037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017881959999999999,"stat_sum":{"num_bytes":19795573,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":12,"num_read":287,"num_read_kb":994,"num_write":166,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"57'126","reported_seq":622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.048837+0000","last_change":"2024-09-16T14:07:11.546511+0000","last_active":"2024-09-16T14:07:51.048837+0000","last_peered":"2024-09-16T14:07:51.048837+0000","last_clean":"2024-09-16T14:07:51.048837+0000","last_became_active":"2024-09-16T14:07:09.559703+0000","last_became_peered":"2024-09-16T14:07:09.559703+0000","last_unstale":"2024-09-16T14:07:51.048837+0000","last_undegraded":"2024-09-16T14:07:51.048837+0000","last_fullsized":"2024-09-16T14:07:51.048837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:06.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074836900000000003,"stat_sum":{"num_bytes":4494935,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":4,"num_read":497,"num_read_kb":788,"num_write":102,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"56'150","reported_seq":690,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.162568+0000","last_change":"2024-09-16T14:07:11.362189+0000","last_active":"2024-09-16T14:07:50.162568+0000","last_peered":"2024-09-16T14:07:50.162568+0000","last_clean":"2024-09-16T14:07:50.162568+0000","last_became_active":"2024-09-16T14:07:09.548973+0000","last_became_peered":"2024-09-16T14:07:09.548973+0000","last_unstale":"2024-09-16T14:07:50.162568+0000","last_undegraded":"2024-09-16T14:07:50.162568+0000","last_fullsized":"2024-09-16T14:07:50.162568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:17.345008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013505660000000001,"stat_sum":{"num_bytes":12107684,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":8,"num_read":1213,"num_read_kb":1029,"num_write":124,"num_write_kb":812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"56'393","reported_seq":2803,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.430158+0000","last_change":"2024-09-16T14:07:11.342735+0000","last_active":"2024-09-16T14:07:50.430158+0000","last_peered":"2024-09-16T14:07:50.430158+0000","last_clean":"2024-09-16T14:07:50.430158+0000","last_became_active":"2024-09-16T14:07:09.559054+0000","last_became_peered":"2024-09-16T14:07:09.559054+0000","last_unstale":"2024-09-16T14:07:50.430158+0000","last_undegraded":"2024-09-16T14:07:50.430158+0000","last_fullsized":"2024-09-16T14:07:50.430158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.926445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372222,"stat_sum":{"num_bytes":35575347,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":4,"num_read":4328,"num_read_kb":9417,"num_write":511,"num_write_kb":5466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"57'175","reported_seq":532,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.121184+0000","last_change":"2024-09-16T14:07:11.340733+0000","last_active":"2024-09-16T14:07:51.121184+0000","last_peered":"2024-09-16T14:07:51.121184+0000","last_clean":"2024-09-16T14:07:51.121184+0000","last_became_active":"2024-09-16T14:07:09.540341+0000","last_became_peered":"2024-09-16T14:07:09.540341+0000","last_unstale":"2024-09-16T14:07:51.121184+0000","last_undegraded":"2024-09-16T14:07:51.121184+0000","last_fullsized":"2024-09-16T14:07:51.121184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:18.953079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012161470000000001,"stat_sum":{"num_bytes":14241201,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":9,"num_read":393,"num_read_kb":4752,"num_write":231,"num_write_kb":3352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"56'302","reported_seq":2231,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.072900+0000","last_change":"2024-09-16T14:07:11.545567+0000","last_active":"2024-09-16T14:07:50.072900+0000","last_peered":"2024-09-16T14:07:50.072900+0000","last_clean":"2024-09-16T14:07:50.072900+0000","last_became_active":"2024-09-16T14:07:09.556146+0000","last_became_peered":"2024-09-16T14:07:09.556146+0000","last_unstale":"2024-09-16T14:07:50.072900+0000","last_undegraded":"2024-09-16T14:07:50.072900+0000","last_fullsized":"2024-09-16T14:07:50.072900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:00:29.300372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037127800000000001,"stat_sum":{"num_bytes":14177913,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":2,"num_read":2661,"num_read_kb":7012,"num_write":384,"num_write_kb":5877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"56'208","reported_seq":874,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T14:07:50.425618+0000","last_change":"2024-09-16T14:07:11.547712+0000","last_active":"2024-09-16T14:07:50.425618+0000","last_peered":"2024-09-16T14:07:50.425618+0000","last_clean":"2024-09-16T14:07:50.425618+0000","last_became_active":"2024-09-16T14:07:09.553092+0000","last_became_peered":"2024-09-16T14:07:09.553092+0000","last_unstale":"2024-09-16T14:07:50.425618+0000","last_undegraded":"2024-09-16T14:07:50.425618+0000","last_fullsized":"2024-09-16T14:07:50.425618+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:54.650445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012662420000000001,"stat_sum":{"num_bytes":18867750,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":4,"num_read":1178,"num_read_kb":1931,"num_write":184,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"57'169","reported_seq":947,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T14:07:51.122837+0000","last_change":"2024-09-16T14:07:11.546794+0000","last_active":"2024-09-16T14:07:51.122837+0000","last_peered":"2024-09-16T14:07:51.122837+0000","last_clean":"2024-09-16T14:07:51.122837+0000","last_became_active":"2024-09-16T14:07:09.555363+0000","last_became_peered":"2024-09-16T14:07:09.555363+0000","last_unstale":"2024-09-16T14:07:51.122837+0000","last_undegraded":"2024-09-16T14:07:51.122837+0000","last_fullsized":"2024-09-16T14:07:51.122837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:36:05.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066482800000000001,"stat_sum":{"num_bytes":30001976,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":7,"num_read":808,"num_read_kb":5773,"num_write":196,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":506171941,"num_objects":1470,"num_object_clones":250,"num_object_copies":2940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1470,"num_whiteouts":268,"num_read":62236,"num_read_kb":153664,"num_write":12632,"num_write_kb":101077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17162240,"data_stored":107730170,"data_compressed":3500720,"data_compressed_allocated":10326016,"data_compressed_original":103575338,"omap_allocated":0,"internal_metadata":0},"log_size":9892,"ondisk_log_size":9892,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307312,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064528,"statfs":{"total":96636764160,"available":96322076672,"internally_reserved":0,"allocated":1667072,"data_stored":4479172,"data_compressed":174898,"data_compressed_allocated":438272,"data_compressed_original":3837952,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]}]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308880,"kb_used_data":3208,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062960,"statfs":{"total":96636764160,"available":96320471040,"internally_reserved":0,"allocated":3284992,"data_stored":20149602,"data_compressed":508079,"data_compressed_allocated":1765376,"data_compressed_original":19380694,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317184,"kb_used_data":3188,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94054656,"statfs":{"total":96636764160,"available":96311967744,"internally_reserved":0,"allocated":3264512,"data_stored":20382793,"data_compressed":479695,"data_compressed_allocated":1703936,"data_compressed_original":19517190,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312696,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059144,"statfs":{"total":96636764160,"available":96316563456,"internally_reserved":0,"allocated":2932736,"data_stored":12159821,"data_compressed":563604,"data_compressed_allocated":1290240,"data_compressed_original":11292164,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312316,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059524,"statfs":{"total":96636764160,"available":96316952576,"internally_reserved":0,"allocated":2539520,"data_stored":7537972,"data_compressed":347665,"data_compressed_allocated":921600,"data_compressed_original":6684349,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312664,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059176,"statfs":{"total":96636764160,"available":96316596224,"internally_reserved":0,"allocated":2891776,"data_stored":16464202,"data_compressed":465149,"data_compressed_allocated":1482752,"data_compressed_original":15699968,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312960,"kb_used_data":3116,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058880,"statfs":{"total":96636764160,"available":96316293120,"internally_reserved":0,"allocated":3190784,"data_stored":14030516,"data_compressed":585319,"data_compressed_allocated":1445888,"data_compressed_original":13057471,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308288,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063552,"statfs":{"total":96636764160,"available":96321077248,"internally_reserved":0,"allocated":2654208,"data_stored":15907296,"data_compressed":356305,"data_compressed_allocated":1314816,"data_compressed_original":15195086,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2539520,"data_stored":13746461,"data_compressed":585319,"data_compressed_allocated":1445888,"data_compressed_original":13057471,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2183168,"data_stored":15589779,"data_compressed":459758,"data_compressed_allocated":1445888,"data_compressed_original":15126528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":7253917,"data_compressed":347665,"data_compressed_allocated":921600,"data_compressed_original":6684349,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2277376,"data_stored":11875747,"data_compressed":563604,"data_compressed_allocated":1290240,"data_compressed_original":11292164,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":15704771,"data_compressed":387095,"data_compressed_allocated":1351680,"data_compressed_original":15264718,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2555904,"data_stored":19508370,"data_compressed":474304,"data_compressed_allocated":1667072,"data_compressed_original":18943750,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":19865528,"data_compressed":508079,"data_compressed_allocated":1765376,"data_compressed_original":19380694,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1015808,"data_stored":4185597,"data_compressed":174896,"data_compressed_allocated":438272,"data_compressed_original":3825664,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:07:52.818 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T14:07:52.818 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:07:52.818 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:07:52.818 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T14:07:52.818 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:07:52.855 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:52.850+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:52.873 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:52.873 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:52.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:52.897 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T14:07:52.897 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-09-16T14:07:52.897 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:07:52.897 DEBUG:teuthology.orchestra.run.smithi079:> type ceph-objectstore-tool 2024-09-16T14:07:52.901 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T14:07:52.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-09-16T14:07:52.956 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:52.954+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:52.971 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:52.971 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:52.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:53.057 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:53.050+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:53.082 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:53.082 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:53.092 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:53.158 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:53.150+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:53.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:07:53.198 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:53.198 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:53.208 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:53.259 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:53.253+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:53.298 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:07:53.308 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:53.308 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:53.317 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:53.318 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:07:53.339 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:07:53.360 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:53.354+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:53.413 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:53.414 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:53.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:53.461 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:53.453+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:53.562 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:53.554+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:53.570 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:53.570 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:53.579 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:53.663 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:53.658+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:53.675 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:53.675 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:53.684 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:53.764 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:53.758+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:53.826 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:53.826 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:53.834 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:53.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:07:53.845+0000 7f3046625300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T14:07:53.864 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:53.857+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:53.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:07:53.932 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:53.933 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:53.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:53.965 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:53.957+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:54.047 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:54.047 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:54.056 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:54.066 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:54.058+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:54.152 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:54.153 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:54.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:54.207 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:54.202+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:54.264 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:54.264 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:54.267 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:54.262+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:54.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:54.368 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:54.362+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:54.378 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:54.378 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:54.387 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:54.469 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:54.462+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:54.479 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:54.479 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:54.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:54.570 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:54.562+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:54.605 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43675 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.906972+0000", 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68029510100000001, 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68033531300000005, 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43675 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906972+0000", 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906972+0000", 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906974+0000", 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8980000000000001e-06 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906981+0000", 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906993+0000", 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2167999999999999e-05 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907000+0000", 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9600000000000003e-06 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908837+0000", 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018372 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908844+0000", 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6640000000000001e-06 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44178 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907817+0000", 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67945018199999996, 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67960306100000001, 2024-09-16T14:07:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44178 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907817+0000", 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907817+0000", 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907818+0000", 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6199999999999998e-07 2024-09-16T14:07:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907820+0000", 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3149999999999999e-06 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907826+0000", 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907831+0000", 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0420000000000002e-06 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909954+0000", 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021232410000000001 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909962+0000", 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1210000000000007e-06 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44446 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908541+0000", 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67872666599999998, 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67897567000000003, 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44446 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908541+0000", 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908541+0000", 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908541+0000", 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4799999999999999e-07 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908543+0000", 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2129999999999998e-06 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908550+0000", 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4880000000000004e-06 2024-09-16T14:07:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908554+0000", 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7869999999999999e-06 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910853+0000", 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022992609999999999 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910856+0000", 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5110000000000002e-06 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44948 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909344+0000", 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67792310200000006, 2024-09-16T14:07:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67826546399999998, 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44948 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909344+0000", 2024-09-16T14:07:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909344+0000", 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909345+0000", 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909346+0000", 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1820000000000001e-06 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909349+0000", 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2720000000000002e-06 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909351+0000", 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.295e-06 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911692+0000", 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00234081 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911694+0000", 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2900000000000001e-06 2024-09-16T14:07:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67720299699999997, 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67763914999999997, 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:07:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43676 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907011+0000", 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68025620399999998, 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68078520799999998, 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43676 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907011+0000", 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907011+0000", 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907012+0000", 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.13e-07 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9220000000000001e-06 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0269999999999998e-06 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907026+0000", 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.386e-06 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908866+0000", 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001840312 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908872+0000", 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44179 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907876+0000", 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.679391941, 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68001328299999997, 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44179 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907876+0000", 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907876+0000", 2024-09-16T14:07:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907876+0000", 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0100000000000004e-07 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907879+0000", 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.965e-06 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907888+0000", 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2930000000000001e-06 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907893+0000", 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3909999999999997e-06 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002101463 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909997+0000", 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6419999999999999e-06 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44447 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908557+0000", 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67871006899999997, 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67942654300000005, 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44447 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908557+0000", 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908557+0000", 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908558+0000", 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908560+0000", 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0420000000000001e-06 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908566+0000", 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3589999999999996e-06 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908571+0000", 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7600000000000002e-06 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910881+0000", 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023100529999999998 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910884+0000", 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5749999999999999e-06 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44949 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909354+0000", 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67791395499999996, 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67872407700000004, 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44949 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909354+0000", 2024-09-16T14:07:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909354+0000", 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909354+0000", 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7399999999999999e-07 2024-09-16T14:07:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909358+0000", 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8410000000000002e-06 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909361+0000", 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3790000000000002e-06 2024-09-16T14:07:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909363+0000", 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2759999999999999e-06 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911716+0000", 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023525360000000001 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911719+0000", 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9799999999999998e-06 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67718040499999999, 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67808310400000005, 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:07:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43677 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907032+0000", 2024-09-16T14:07:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68023567699999998, 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68123131400000003, 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43677 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907032+0000", 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907032+0000", 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907032+0000", 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6400000000000003e-07 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907035+0000", 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3649999999999998e-06 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907041+0000", 2024-09-16T14:07:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2859999999999996e-06 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907045+0000", 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0559999999999998e-06 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908894+0000", 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018492389999999999 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908899+0000", 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8670000000000002e-06 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44180 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907897+0000", 2024-09-16T14:07:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67937011199999997, 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68045754700000005, 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44180 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907897+0000", 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907897+0000", 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907898+0000", 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.92e-07 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907900+0000", 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.322e-06 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907907+0000", 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907910+0000", 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7450000000000001e-06 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910028+0000", 2024-09-16T14:07:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021173300000000002 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910030+0000", 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5019999999999999e-06 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44448 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908575+0000", 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67869226999999999, 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.679871793, 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44448 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908575+0000", 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908575+0000", 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908576+0000", 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2e-07 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908578+0000", 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.108e-06 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908584+0000", 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3199999999999996e-06 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908590+0000", 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6420000000000004e-06 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910911+0000", 2024-09-16T14:07:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002321095 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910913+0000", 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6709999999999999e-06 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44950 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909379+0000", 2024-09-16T14:07:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67788868300000005, 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67916002399999997, 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44950 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909379+0000", 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909379+0000", 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909379+0000", 2024-09-16T14:07:54.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6800000000000002e-07 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909380+0000", 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.269e-06 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909384+0000", 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6480000000000001e-06 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909386+0000", 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.26e-06 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911739+0000", 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023528030000000001 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911741+0000", 2024-09-16T14:07:54.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3159999999999999e-06 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67715133299999997, 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67851936499999999, 2024-09-16T14:07:54.676 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:54.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:54.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:07:54.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43678 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68021759199999998, 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68167727499999997, 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43678 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6699999999999999e-07 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.311e-06 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907059+0000", 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2190000000000001e-06 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907063+0000", 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1649999999999999e-06 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908923+0000", 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001859882 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908928+0000", 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9729999999999997e-06 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44181 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907916+0000", 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67935180900000003, 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68090314399999996, 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44181 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907916+0000", 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907916+0000", 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907916+0000", 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7700000000000004e-07 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907919+0000", 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3989999999999998e-06 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907925+0000", 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5899999999999996e-06 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907929+0000", 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0829999999999997e-06 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002135943 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910068+0000", 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5959999999999998e-06 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44449 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908596+0000", 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67867187200000001, 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68031585100000003, 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44449 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908596+0000", 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908596+0000", 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908596+0000", 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.27e-07 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908598+0000", 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.17e-06 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908607+0000", 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6670000000000002e-06 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908612+0000", 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7199999999999997e-06 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910937+0000", 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023253929999999998 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910940+0000", 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5909999999999998e-06 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44951 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909405+0000", 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67786229200000003, 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67959797899999996, 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44951 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909405+0000", 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909405+0000", 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909405+0000", 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7799999999999997e-07 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909407+0000", 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5230000000000001e-06 2024-09-16T14:07:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909411+0000", 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5470000000000001e-06 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909413+0000", 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9610000000000001e-06 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911763+0000", 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023497040000000002 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911766+0000", 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0019999999999998e-06 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67713298700000002, 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67896030200000002, 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:07:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43679 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907067+0000", 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68020033899999999, 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68211982599999998, 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43679 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907067+0000", 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907067+0000", 2024-09-16T14:07:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907068+0000", 2024-09-16T14:07:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:07:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907070+0000", 2024-09-16T14:07:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2510000000000001e-06 2024-09-16T14:07:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907076+0000", 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907079+0000", 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6729999999999998e-06 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908961+0000", 2024-09-16T14:07:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001881085 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908966+0000", 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5559999999999998e-06 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44182 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907934+0000", 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67933395399999996, 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68134511499999995, 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44182 2024-09-16T14:07:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907934+0000", 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907934+0000", 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907934+0000", 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8100000000000003e-07 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907940+0000", 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5480000000000001e-06 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907946+0000", 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9160000000000004e-06 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907950+0000", 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-06 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002146078 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6170000000000001e-06 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44450 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908616+0000", 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67865190399999997, 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68075506699999999, 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44450 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908616+0000", 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908616+0000", 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908616+0000", 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908618+0000", 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908634+0000", 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5665999999999999e-05 2024-09-16T14:07:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908638+0000", 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.749e-06 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910964+0000", 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023263120000000001 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910967+0000", 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6450000000000001e-06 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44952 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909416+0000", 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67785187099999999, 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68005341100000005, 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44952 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909416+0000", 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909416+0000", 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909416+0000", 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6800000000000002e-07 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909417+0000", 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.122e-06 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909420+0000", 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3189999999999999e-06 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909423+0000", 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3929999999999998e-06 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911788+0000", 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002365799 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911791+0000", 2024-09-16T14:07:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3369999999999998e-06 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67711564800000001, 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67942360599999996, 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:07:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43680 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68018163899999995, 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68258468100000003, 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43680 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.46e-07 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907088+0000", 2024-09-16T14:07:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1789999999999998e-06 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907096+0000", 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6410000000000002e-06 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907100+0000", 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2119999999999997e-06 2024-09-16T14:07:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909001+0000", 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019009750000000001 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909007+0000", 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.993e-06 2024-09-16T14:07:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44183 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907994+0000", 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67927326700000001, 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68176772399999996, 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44183 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907994+0000", 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907994+0000", 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907995+0000", 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8400000000000003e-07 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907998+0000", 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2459999999999999e-06 2024-09-16T14:07:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908007+0000", 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8899999999999996e-06 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908012+0000", 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.814e-06 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910126+0000", 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002114318 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8240000000000001e-06 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44451 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908657+0000", 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67861045399999997, 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68120043900000005, 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44451 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908657+0000", 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908657+0000", 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908658+0000", 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8999999999999995e-07 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908663+0000", 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0690000000000001e-06 2024-09-16T14:07:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908677+0000", 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3986e-05 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908684+0000", 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6309999999999997e-06 2024-09-16T14:07:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910990+0000", 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002306532 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910993+0000", 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4549999999999998e-06 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44953 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909438+0000", 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67782946200000005, 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68058259600000004, 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44953 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909438+0000", 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909438+0000", 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909438+0000", 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7700000000000001e-07 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909440+0000", 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4109999999999999e-06 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909443+0000", 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.737e-06 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909446+0000", 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3080000000000002e-06 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911814+0000", 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023681269999999998 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911817+0000", 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1329999999999999e-06 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67709972500000004, 2024-09-16T14:07:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.67994586400000001, 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43681 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907105+0000", 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68016228599999995, 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68310403600000003, 2024-09-16T14:07:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43681 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907105+0000", 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907105+0000", 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907106+0000", 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2600000000000002e-07 2024-09-16T14:07:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3599999999999999e-06 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8420000000000001e-06 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907119+0000", 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4809999999999997e-06 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909036+0000", 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019165390000000001 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909039+0000", 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44184 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908027+0000", 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.679240709, 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68227445099999995, 2024-09-16T14:07:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44184 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908027+0000", 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908027+0000", 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908028+0000", 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0999999999999997e-07 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908030+0000", 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8250000000000001e-06 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908037+0000", 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0199999999999997e-06 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908042+0000", 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1359999999999999e-06 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910153+0000", 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002111941 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910156+0000", 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.644e-06 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44452 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908689+0000", 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67857823699999997, 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68170715299999995, 2024-09-16T14:07:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44452 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908689+0000", 2024-09-16T14:07:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908689+0000", 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908690+0000", 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7700000000000004e-07 2024-09-16T14:07:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908693+0000", 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6769999999999999e-06 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908699+0000", 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6200000000000001e-06 2024-09-16T14:07:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908704+0000", 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2629999999999997e-06 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911015+0000", 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002311727 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911018+0000", 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5320000000000001e-06 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44954 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909465+0000", 2024-09-16T14:07:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67780237799999998, 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68102265200000001, 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44954 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909465+0000", 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909465+0000", 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909465+0000", 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909467+0000", 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2979999999999999e-06 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909470+0000", 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.574e-06 2024-09-16T14:07:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909481+0000", 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0426e-05 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911837+0000", 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002356131 2024-09-16T14:07:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911839+0000", 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4140000000000001e-06 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43682 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907124+0000", 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68014362399999995, 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68345961600000005, 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43682 2024-09-16T14:07:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907124+0000", 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907124+0000", 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907124+0000", 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.32e-07 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907126+0000", 2024-09-16T14:07:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1090000000000001e-06 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907133+0000", 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4649999999999999e-06 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0080000000000004e-06 2024-09-16T14:07:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909070+0000", 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019334669999999999 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909076+0000", 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2700000000000004e-06 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44185 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908074+0000", 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67919321700000002, 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68260472999999999, 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44185 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908074+0000", 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908074+0000", 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908075+0000", 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908078+0000", 2024-09-16T14:07:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0589999999999998e-06 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908085+0000", 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9550000000000003e-06 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908089+0000", 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1200000000000004e-06 2024-09-16T14:07:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910177+0000", 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020877220000000002 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910179+0000", 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.509e-06 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44688 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908707+0000", 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67855998200000001, 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68206317299999997, 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44688 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908707+0000", 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908707+0000", 2024-09-16T14:07:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908709+0000", 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.66e-06 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908711+0000", 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0849999999999999e-06 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908719+0000", 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5340000000000002e-06 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908723+0000", 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7960000000000002e-06 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911043+0000", 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002320623 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911046+0000", 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.23e-06 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44955 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909485+0000", 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67778294500000003, 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68138160000000003, 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44955 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909485+0000", 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909485+0000", 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909485+0000", 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5900000000000003e-07 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909487+0000", 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1720000000000001e-06 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909492+0000", 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5179999999999998e-06 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909494+0000", 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911860+0000", 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023663239999999999 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911863+0000", 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.356e-06 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43918 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907141+0000", 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68012649999999997, 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68381909699999999, 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43918 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907141+0000", 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907141+0000", 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907141+0000", 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1699999999999999e-07 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907143+0000", 2024-09-16T14:07:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0559999999999999e-06 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907151+0000", 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1579999999999999e-06 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0570000000000003e-06 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909127+0000", 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019723409999999999 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909133+0000", 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1469999999999998e-06 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44186 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908104+0000", 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67916337800000004, 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68295101899999999, 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44186 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908104+0000", 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908104+0000", 2024-09-16T14:07:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908105+0000", 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7700000000000004e-07 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908107+0000", 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7219999999999999e-06 2024-09-16T14:07:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908115+0000", 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8569999999999995e-06 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908119+0000", 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8519999999999997e-06 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910207+0000", 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020876660000000002 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910209+0000", 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4789999999999999e-06 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44689 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908736+0000", 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67853195200000005, 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68241150800000006, 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44689 2024-09-16T14:07:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908736+0000", 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908736+0000", 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908736+0000", 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8800000000000002e-07 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908743+0000", 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.348e-06 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908749+0000", 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4330000000000001e-06 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908753+0000", 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7929999999999999e-06 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911072+0000", 2024-09-16T14:07:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00231883 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911074+0000", 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.526e-06 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44956 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909515+0000", 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67775269000000005, 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68172754400000002, 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44956 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909515+0000", 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909515+0000", 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909515+0000", 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909516+0000", 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3599999999999999e-06 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909521+0000", 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1980000000000003e-06 2024-09-16T14:07:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909523+0000", 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2520000000000002e-06 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911884+0000", 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023610789999999999 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911886+0000", 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4370000000000001e-06 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43919 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907161+0000", 2024-09-16T14:07:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.680106341, 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.684176636, 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43919 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907161+0000", 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907161+0000", 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907162+0000", 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1800000000000001e-07 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2649999999999999e-06 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1120000000000001e-06 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907174+0000", 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6710000000000001e-06 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909174+0000", 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020007380000000002 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909179+0000", 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1240000000000004e-06 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44187 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908138+0000", 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67912904699999999, 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68329137399999995, 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44187 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908138+0000", 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908138+0000", 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908139+0000", 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2900000000000003e-07 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908142+0000", 2024-09-16T14:07:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6929999999999999e-06 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908150+0000", 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908154+0000", 2024-09-16T14:07:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.233e-06 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910232+0000", 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020781559999999998 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910234+0000", 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5119999999999998e-06 2024-09-16T14:07:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44690 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908767+0000", 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67850068900000005, 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68275801300000005, 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44690 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908767+0000", 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908767+0000", 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908767+0000", 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3900000000000004e-07 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908769+0000", 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.075e-06 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908775+0000", 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7910000000000001e-06 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908780+0000", 2024-09-16T14:07:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7269999999999998e-06 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911097+0000", 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002316668 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911099+0000", 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5160000000000001e-06 2024-09-16T14:07:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44957 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909536+0000", 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67773148699999997, 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.682080456, 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44957 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909536+0000", 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909536+0000", 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909536+0000", 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9299999999999999e-07 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909538+0000", 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.336e-06 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909541+0000", 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6270000000000002e-06 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909544+0000", 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2859999999999998e-06 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911905+0000", 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023612199999999998 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911907+0000", 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3609999999999999e-06 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43920 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907178+0000", 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68008921700000002, 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68453794000000001, 2024-09-16T14:07:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43920 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907178+0000", 2024-09-16T14:07:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907178+0000", 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-16T14:07:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907181+0000", 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2019999999999998e-06 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907187+0000", 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8549999999999998e-06 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907190+0000", 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909220+0000", 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020297269999999998 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909225+0000", 2024-09-16T14:07:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.994e-06 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44188 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908158+0000", 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67910927899999995, 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.683649693, 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44188 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908158+0000", 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908158+0000", 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908159+0000", 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908161+0000", 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.322e-06 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908167+0000", 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4019999999999999e-06 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908171+0000", 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0090000000000001e-06 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910261+0000", 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002089254 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910263+0000", 2024-09-16T14:07:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5959999999999998e-06 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44691 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908795+0000", 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67847262200000003, 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68311197099999998, 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44691 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908795+0000", 2024-09-16T14:07:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908795+0000", 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908795+0000", 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5199999999999996e-07 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908798+0000", 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1280000000000002e-06 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908804+0000", 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1519999999999997e-06 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908808+0000", 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911123+0000", 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023157820000000002 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911126+0000", 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5339999999999998e-06 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44958 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909547+0000", 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67772038899999998, 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68245498299999996, 2024-09-16T14:07:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44958 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909547+0000", 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909547+0000", 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909547+0000", 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6e-07 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909549+0000", 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1999999999999999e-06 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909552+0000", 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2279999999999998e-06 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909554+0000", 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2970000000000002e-06 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911929+0000", 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002374686 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911931+0000", 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43921 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68007352899999995, 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68490130699999996, 2024-09-16T14:07:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43921 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907197+0000", 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2500000000000001e-06 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907203+0000", 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9839999999999998e-06 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907206+0000", 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6500000000000002e-06 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909259+0000", 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002053086 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909263+0000", 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3749999999999999e-06 2024-09-16T14:07:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44189 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908175+0000", 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67909216400000005, 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68401553699999995, 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44189 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908175+0000", 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908175+0000", 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908176+0000", 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7599999999999997e-07 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908178+0000", 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1909999999999999e-06 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908185+0000", 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5459999999999997e-06 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908188+0000", 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.873e-06 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910286+0000", 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002097537 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910289+0000", 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5749999999999999e-06 2024-09-16T14:07:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44692 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908830+0000", 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67843795200000001, 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68345273900000003, 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44692 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908830+0000", 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908830+0000", 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908830+0000", 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1799999999999995e-07 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908832+0000", 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1959999999999998e-06 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908838+0000", 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9699999999999996e-06 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908842+0000", 2024-09-16T14:07:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7440000000000001e-06 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911149+0000", 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023065350000000002 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911151+0000", 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5679999999999998e-06 2024-09-16T14:07:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44959 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909557+0000", 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67771043500000006, 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68282078999999996, 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44959 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909557+0000", 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909557+0000", 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909557+0000", 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909558+0000", 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909562+0000", 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2569999999999999e-06 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909564+0000", 2024-09-16T14:07:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3970000000000001e-06 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911952+0000", 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023877740000000001 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911954+0000", 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.413e-06 2024-09-16T14:07:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43922 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907210+0000", 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68005710100000005, 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68525938399999997, 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43922 2024-09-16T14:07:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907210+0000", 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907210+0000", 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907211+0000", 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1800000000000001e-07 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907217+0000", 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3910000000000003e-06 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907224+0000", 2024-09-16T14:07:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9979999999999997e-06 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907228+0000", 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0500000000000002e-06 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909292+0000", 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020637889999999999 2024-09-16T14:07:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909296+0000", 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2980000000000002e-06 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44190 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908193+0000", 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67907465300000003, 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68437210000000004, 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44190 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908193+0000", 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908193+0000", 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908193+0000", 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1699999999999999e-07 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908195+0000", 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.227e-06 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908202+0000", 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908205+0000", 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.569e-06 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910314+0000", 2024-09-16T14:07:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021089949999999998 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910317+0000", 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5809999999999999e-06 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44693 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908846+0000", 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67842163099999997, 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68394502000000001, 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44693 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908846+0000", 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908846+0000", 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908846+0000", 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5400000000000002e-07 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908848+0000", 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.734e-06 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908854+0000", 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908857+0000", 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6799999999999999e-06 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911177+0000", 2024-09-16T14:07:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023197500000000002 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911180+0000", 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5639999999999999e-06 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44960 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909585+0000", 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67768203900000001, 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68329915900000004, 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44960 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909585+0000", 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909585+0000", 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909586+0000", 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909587+0000", 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.249e-06 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909590+0000", 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5439999999999999e-06 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909593+0000", 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.278e-06 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911976+0000", 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023827840000000002 2024-09-16T14:07:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911978+0000", 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3120000000000001e-06 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43923 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907235+0000", 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68003272699999995, 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68574563899999996, 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43923 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907235+0000", 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907235+0000", 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907235+0000", 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4899999999999995e-07 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907242+0000", 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0759999999999997e-06 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907249+0000", 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8000000000000001e-06 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907253+0000", 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1359999999999999e-06 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909322+0000", 2024-09-16T14:07:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020689710000000002 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909325+0000", 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.632e-06 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44191 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908209+0000", 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67905861000000001, 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68486326600000003, 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44191 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908209+0000", 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908209+0000", 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908209+0000", 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908212+0000", 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4030000000000001e-06 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908218+0000", 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1279999999999996e-06 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908221+0000", 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.252e-06 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910341+0000", 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021203659999999998 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910344+0000", 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6879999999999999e-06 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44694 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908861+0000", 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67840622799999994, 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68430627899999996, 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44694 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908861+0000", 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908861+0000", 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908862+0000", 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908867+0000", 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9169999999999997e-06 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908872+0000", 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.677e-06 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908876+0000", 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.518e-06 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911202+0000", 2024-09-16T14:07:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023264169999999999 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911205+0000", 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4980000000000001e-06 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44961 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909613+0000", 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67765440700000001, 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68364607600000005, 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44961 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909613+0000", 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909613+0000", 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909613+0000", 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909620+0000", 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3790000000000003e-06 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909623+0000", 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7309999999999999e-06 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909626+0000", 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2510000000000001e-06 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911998+0000", 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023725389999999999 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912001+0000", 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3630000000000001e-06 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43924 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907258+0000", 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.68000947300000003, 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68609679999999995, 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43924 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907258+0000", 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907258+0000", 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907258+0000", 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4099999999999999e-07 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5210000000000001e-06 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907268+0000", 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0330000000000003e-06 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907273+0000", 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.967e-06 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909350+0000", 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020771280000000001 2024-09-16T14:07:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909355+0000", 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.527e-06 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44192 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908225+0000", 2024-09-16T14:07:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67904273800000003, 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68522600899999997, 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44192 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908225+0000", 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908225+0000", 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908225+0000", 2024-09-16T14:07:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908227+0000", 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0710000000000002e-06 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908233+0000", 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6960000000000002e-06 2024-09-16T14:07:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908236+0000", 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0790000000000001e-06 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910367+0000", 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021306200000000002 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910369+0000", 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.554e-06 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44695 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908881+0000", 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67838607200000001, 2024-09-16T14:07:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68466871100000004, 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44695 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908881+0000", 2024-09-16T14:07:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908881+0000", 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908882+0000", 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8099999999999998e-07 2024-09-16T14:07:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908884+0000", 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9769999999999999e-06 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908889+0000", 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5979999999999996e-06 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908893+0000", 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3680000000000002e-06 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911230+0000", 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023377559999999999 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911233+0000", 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5959999999999998e-06 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44962 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909646+0000", 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67762167399999995, 2024-09-16T14:07:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.684000151, 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44962 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909646+0000", 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909646+0000", 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909646+0000", 2024-09-16T14:07:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909647+0000", 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2330000000000001e-06 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909651+0000", 2024-09-16T14:07:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5860000000000001e-06 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909653+0000", 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912023+0000", 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023696640000000001 2024-09-16T14:07:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912025+0000", 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3319999999999999e-06 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43925 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907277+0000", 2024-09-16T14:07:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67999064399999998, 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68646135200000002, 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43925 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907277+0000", 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907277+0000", 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907277+0000", 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907279+0000", 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2000000000000001e-06 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907285+0000", 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9279999999999998e-06 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907291+0000", 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6099999999999997e-06 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909379+0000", 2024-09-16T14:07:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002087981 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909382+0000", 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7149999999999998e-06 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44193 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908239+0000", 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67902805799999999, 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68559381699999999, 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44193 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908239+0000", 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908239+0000", 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908240+0000", 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5199999999999998e-07 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908242+0000", 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9630000000000001e-06 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908247+0000", 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6810000000000003e-06 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908251+0000", 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4460000000000001e-06 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910389+0000", 2024-09-16T14:07:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021379950000000002 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910391+0000", 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6019999999999998e-06 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44696 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908896+0000", 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67837099700000003, 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68502871099999996, 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44696 2024-09-16T14:07:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908896+0000", 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908896+0000", 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908897+0000", 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908899+0000", 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7999999999999999e-06 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908904+0000", 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7150000000000003e-06 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908908+0000", 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5140000000000002e-06 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911256+0000", 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002348054 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911259+0000", 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5849999999999998e-06 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44963 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909673+0000", 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67759449800000005, 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68435049999999997, 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44963 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909673+0000", 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909673+0000", 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909673+0000", 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909675+0000", 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2699999999999999e-06 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909678+0000", 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5750000000000001e-06 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909681+0000", 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5110000000000002e-06 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912045+0000", 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023644590000000002 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912047+0000", 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3659999999999999e-06 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43926 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907306+0000", 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67996188999999996, 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68681337600000003, 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43926 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907306+0000", 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907306+0000", 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907306+0000", 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8999999999999997e-07 2024-09-16T14:07:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907309+0000", 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.644e-06 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907316+0000", 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9430000000000003e-06 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907320+0000", 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4880000000000001e-06 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909408+0000", 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020882700000000001 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909414+0000", 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.959e-06 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44194 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908256+0000", 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67901187900000004, 2024-09-16T14:07:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68595508999999999, 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44194 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908256+0000", 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908256+0000", 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908256+0000", 2024-09-16T14:07:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8000000000000001e-07 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908262+0000", 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0850000000000002e-06 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908269+0000", 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6839999999999999e-06 2024-09-16T14:07:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908272+0000", 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6959999999999998e-06 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910419+0000", 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002146181 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910421+0000", 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6029999999999999e-06 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44697 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908933+0000", 2024-09-16T14:07:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67833486099999996, 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68537307599999997, 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44697 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908933+0000", 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908933+0000", 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908933+0000", 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5999999999999999e-07 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908935+0000", 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9740000000000001e-06 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908941+0000", 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0059999999999998e-06 2024-09-16T14:07:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908945+0000", 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4689999999999999e-06 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911284+0000", 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023389309999999998 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911287+0000", 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6029999999999999e-06 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44964 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909683+0000", 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67758476099999998, 2024-09-16T14:07:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68471447100000005, 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44964 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909683+0000", 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909683+0000", 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909683+0000", 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6600000000000003e-07 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909684+0000", 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.192e-06 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909687+0000", 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2049999999999998e-06 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909690+0000", 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.244e-06 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912071+0000", 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002380938 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912073+0000", 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3630000000000001e-06 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43927 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907324+0000", 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.679943148, 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68716851899999998, 2024-09-16T14:07:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43927 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907324+0000", 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907324+0000", 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907325+0000", 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2599999999999998e-07 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907327+0000", 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3659999999999999e-06 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907335+0000", 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6650000000000003e-06 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907339+0000", 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.036e-06 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909438+0000", 2024-09-16T14:07:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020994120000000002 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909441+0000", 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6460000000000002e-06 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44195 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908277+0000", 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67899072000000005, 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68630778599999998, 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44195 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908277+0000", 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908277+0000", 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908277+0000", 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.63e-07 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908279+0000", 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908286+0000", 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3770000000000001e-06 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908290+0000", 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.861e-06 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910443+0000", 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021534670000000001 2024-09-16T14:07:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910446+0000", 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.633e-06 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44698 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908957+0000", 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67831088100000003, 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68572287099999996, 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44698 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908957+0000", 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908957+0000", 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908957+0000", 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908959+0000", 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2290000000000002e-06 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908967+0000", 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6559999999999992e-06 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908971+0000", 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7110000000000001e-06 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911311+0000", 2024-09-16T14:07:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023400819999999998 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911313+0000", 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6120000000000001e-06 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45200 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909706+0000", 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67756194800000002, 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68506942999999998, 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45200 2024-09-16T14:07:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909706+0000", 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909706+0000", 2024-09-16T14:07:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909706+0000", 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.53e-07 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909707+0000", 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.234e-06 2024-09-16T14:07:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909711+0000", 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5109999999999999e-06 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909713+0000", 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912094+0000", 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023808039999999998 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912096+0000", 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3439999999999999e-06 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43928 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907343+0000", 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67992475200000002, 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68752427500000002, 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43928 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907343+0000", 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907343+0000", 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907343+0000", 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907345+0000", 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2790000000000001e-06 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907351+0000", 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0440000000000001e-06 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907356+0000", 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6940000000000003e-06 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909469+0000", 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002113167 2024-09-16T14:07:55.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909485+0000", 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5299999999999999e-05 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44432 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908299+0000", 2024-09-16T14:07:55.006 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67896818999999997, 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68666277099999995, 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44432 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908299+0000", 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908299+0000", 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908300+0000", 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908302+0000", 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5720000000000001e-06 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908309+0000", 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1110000000000002e-06 2024-09-16T14:07:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908313+0000", 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.811e-06 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910470+0000", 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021568899999999998 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910473+0000", 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5620000000000002e-06 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44699 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908994+0000", 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67827334100000003, 2024-09-16T14:07:55.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68605948000000005, 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44699 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908994+0000", 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908994+0000", 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908994+0000", 2024-09-16T14:07:55.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908997+0000", 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0880000000000002e-06 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909004+0000", 2024-09-16T14:07:55.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3039999999999999e-06 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909008+0000", 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.811e-06 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911337+0000", 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023294549999999998 2024-09-16T14:07:55.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911340+0000", 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.599e-06 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.016 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67753413699999998, 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68541615300000003, 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:07:55.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43929 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907360+0000", 2024-09-16T14:07:55.021 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67990756100000005, 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68788159800000004, 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43929 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907360+0000", 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907360+0000", 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907360+0000", 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0200000000000003e-07 2024-09-16T14:07:55.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907363+0000", 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907369+0000", 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907372+0000", 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6389999999999998e-06 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909517+0000", 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021447200000000001 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909520+0000", 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5470000000000001e-06 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44433 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908318+0000", 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67894995800000002, 2024-09-16T14:07:55.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68701920299999997, 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44433 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908318+0000", 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908318+0000", 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908318+0000", 2024-09-16T14:07:55.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1399999999999997e-07 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908320+0000", 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908326+0000", 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0129999999999999e-06 2024-09-16T14:07:55.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908330+0000", 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910497+0000", 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002167272 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910500+0000", 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.655e-06 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44700 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909011+0000", 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67825610199999997, 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68642037199999995, 2024-09-16T14:07:55.031 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44700 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909011+0000", 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909011+0000", 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909012+0000", 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7500000000000001e-07 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909014+0000", 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9290000000000001e-06 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909019+0000", 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6439999999999997e-06 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909023+0000", 2024-09-16T14:07:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5449999999999999e-06 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911366+0000", 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002343347 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911369+0000", 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.649e-06 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67752440999999997, 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68578056499999995, 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:07:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:07:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43930 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907376+0000", 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67989101900000004, 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68824209800000002, 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43930 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907376+0000", 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907376+0000", 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907377+0000", 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8099999999999998e-07 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907379+0000", 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2589999999999999e-06 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907385+0000", 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3539999999999996e-06 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907389+0000", 2024-09-16T14:07:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9129999999999996e-06 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909554+0000", 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021646489999999998 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909559+0000", 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6280000000000004e-06 2024-09-16T14:07:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44434 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908334+0000", 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67893361500000005, 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68737632199999998, 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44434 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908334+0000", 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908334+0000", 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908334+0000", 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0400000000000002e-07 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908336+0000", 2024-09-16T14:07:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:07:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908342+0000", 2024-09-16T14:07:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9100000000000002e-06 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908345+0000", 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1580000000000001e-06 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910525+0000", 2024-09-16T14:07:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021792209999999998 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910527+0000", 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5440000000000001e-06 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44701 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909036+0000", 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67823151500000001, 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68676910999999996, 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44701 2024-09-16T14:07:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909036+0000", 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909036+0000", 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909036+0000", 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909038+0000", 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9630000000000001e-06 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909045+0000", 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5220000000000004e-06 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909049+0000", 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6880000000000001e-06 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911392+0000", 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023438080000000002 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911395+0000", 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3630000000000001e-06 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67750084700000002, 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68613007999999998, 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:07:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43931 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907393+0000", 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67987436000000001, 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68859932800000001, 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43931 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907393+0000", 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907393+0000", 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907393+0000", 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907396+0000", 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.165e-06 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907403+0000", 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3100000000000003e-06 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907407+0000", 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2169999999999996e-06 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909586+0000", 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002178765 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909589+0000", 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5629999999999999e-06 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44435 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908349+0000", 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67891868499999997, 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68773833399999995, 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44435 2024-09-16T14:07:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908349+0000", 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908349+0000", 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908349+0000", 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7099999999999997e-07 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908351+0000", 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1160000000000002e-06 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908357+0000", 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9830000000000001e-06 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908361+0000", 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.687e-06 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910554+0000", 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021927079999999998 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910556+0000", 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5749999999999999e-06 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44702 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909070+0000", 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67819793799999994, 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68711027099999999, 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44702 2024-09-16T14:07:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909070+0000", 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909070+0000", 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909070+0000", 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909072+0000", 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3029999999999998e-06 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909079+0000", 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4640000000000003e-06 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909082+0000", 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7450000000000001e-06 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911431+0000", 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002348971 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911434+0000", 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4760000000000001e-06 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.677473608, 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68648096599999997, 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:07:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:07:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:07:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43932 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907411+0000", 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67985629199999997, 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68895577100000005, 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43932 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907411+0000", 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907411+0000", 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907412+0000", 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.89e-07 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907414+0000", 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1459999999999999e-06 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907422+0000", 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907426+0000", 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8380000000000004e-06 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909617+0000", 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021907070000000001 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909621+0000", 2024-09-16T14:07:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8099999999999997e-06 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44436 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908365+0000", 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67890282000000002, 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68810122399999996, 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44436 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908365+0000", 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908365+0000", 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908365+0000", 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5600000000000001e-07 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908367+0000", 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3140000000000002e-06 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908373+0000", 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7930000000000003e-06 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908377+0000", 2024-09-16T14:07:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4960000000000001e-06 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910581+0000", 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002204293 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910583+0000", 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.571e-06 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44703 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909121+0000", 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67814687900000004, 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68743670400000001, 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44703 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909121+0000", 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909121+0000", 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909121+0000", 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2399999999999999e-07 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909123+0000", 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909131+0000", 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0490000000000004e-06 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909135+0000", 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:07:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911456+0000", 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002321006 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911459+0000", 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67746419499999999, 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68684950099999997, 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:07:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:07:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43933 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907430+0000", 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67983744499999998, 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.689318192, 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43933 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907430+0000", 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907430+0000", 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907430+0000", 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1300000000000001e-07 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5009999999999999e-06 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907439+0000", 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5610000000000004e-06 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907443+0000", 2024-09-16T14:07:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5370000000000002e-06 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909646+0000", 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022030230000000001 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909649+0000", 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6199999999999999e-06 2024-09-16T14:07:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44437 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908380+0000", 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.678887083, 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68845937199999996, 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44437 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908380+0000", 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908380+0000", 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908381+0000", 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908383+0000", 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908389+0000", 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.173e-06 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908392+0000", 2024-09-16T14:07:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2540000000000001e-06 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910610+0000", 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022172149999999998 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910612+0000", 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6819999999999999e-06 2024-09-16T14:07:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44704 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909140+0000", 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67812767699999998, 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68779513299999995, 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44704 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909140+0000", 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909140+0000", 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909140+0000", 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7899999999999999e-07 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909142+0000", 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909148+0000", 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6890000000000001e-06 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909151+0000", 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5489999999999998e-06 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911483+0000", 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023312459999999999 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911485+0000", 2024-09-16T14:07:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.424e-06 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.677439876, 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68719918400000002, 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:07:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:07:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43934 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907447+0000", 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67982024600000002, 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68968600099999999, 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43934 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907447+0000", 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907447+0000", 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907448+0000", 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9499999999999998e-07 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907454+0000", 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6059999999999999e-06 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1659999999999999e-06 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907466+0000", 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2870000000000004e-06 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909677+0000", 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022112550000000001 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909681+0000", 2024-09-16T14:07:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4819999999999996e-06 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44438 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908397+0000", 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67887075799999996, 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68882865100000001, 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44438 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908397+0000", 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908397+0000", 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908397+0000", 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3999999999999997e-07 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908399+0000", 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.03e-06 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908405+0000", 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.964e-06 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908408+0000", 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1250000000000001e-06 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910636+0000", 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022282650000000001 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910639+0000", 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.424e-06 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44705 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909168+0000", 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.678099165, 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68815199999999999, 2024-09-16T14:07:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44705 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909168+0000", 2024-09-16T14:07:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909168+0000", 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909169+0000", 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6899999999999998e-07 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909171+0000", 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909177+0000", 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909181+0000", 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7579999999999998e-06 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911506+0000", 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023252120000000001 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911509+0000", 2024-09-16T14:07:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.339e-06 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67742027000000005, 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68756462600000001, 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43935 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907470+0000", 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67979745400000002, 2024-09-16T14:07:55.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.690046681, 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43935 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907470+0000", 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907470+0000", 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907471+0000", 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9800000000000004e-07 2024-09-16T14:07:55.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907473+0000", 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8540000000000001e-06 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907481+0000", 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9840000000000001e-06 2024-09-16T14:07:55.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907485+0000", 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8639999999999998e-06 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909706+0000", 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002220882 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909709+0000", 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.326e-06 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44439 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908412+0000", 2024-09-16T14:07:55.147 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67885590799999995, 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68920016900000003, 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44439 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908412+0000", 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908412+0000", 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908412+0000", 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908418+0000", 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4659999999999996e-06 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908424+0000", 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8549999999999998e-06 2024-09-16T14:07:55.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908428+0000", 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.036e-06 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910665+0000", 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022369339999999999 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910668+0000", 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6079999999999998e-06 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44706 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909185+0000", 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67808213100000003, 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68851802299999998, 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.152 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44706 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909185+0000", 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909185+0000", 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909186+0000", 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2399999999999999e-07 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909191+0000", 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6470000000000003e-06 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909197+0000", 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5419999999999996e-06 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909200+0000", 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5860000000000001e-06 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911531+0000", 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023300740000000001 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911533+0000", 2024-09-16T14:07:55.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3379999999999999e-06 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67740008200000001, 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68792394300000004, 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.157 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:07:55.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43936 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907490+0000", 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67977769899999996, 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.69033329499999996, 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.162 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43936 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907490+0000", 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907490+0000", 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907490+0000", 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907492+0000", 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.215e-06 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907498+0000", 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8780000000000003e-06 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907507+0000", 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0750000000000004e-06 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909737+0000", 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022295869999999999 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909742+0000", 2024-09-16T14:07:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2669999999999997e-06 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44440 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908432+0000", 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67883559900000001, 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68942400699999995, 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.167 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44440 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908432+0000", 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908432+0000", 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908432+0000", 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908434+0000", 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0449999999999999e-06 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908441+0000", 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2960000000000004e-06 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908449+0000", 2024-09-16T14:07:55.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9480000000000008e-06 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910689+0000", 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022407630000000002 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910692+0000", 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6860000000000002e-06 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.171 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44707 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909213+0000", 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67805461099999997, 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68877408600000001, 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44707 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909213+0000", 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909213+0000", 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909213+0000", 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7800000000000002e-07 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909215+0000", 2024-09-16T14:07:55.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.119e-06 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909221+0000", 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0789999999999997e-06 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909228+0000", 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6100000000000002e-06 2024-09-16T14:07:55.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911553+0000", 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023252070000000001 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911556+0000", 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.362e-06 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.176 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67737681299999997, 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68813068600000005, 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43937 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907514+0000", 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67975390199999997, 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.690541179, 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.182 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43937 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907514+0000", 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907514+0000", 2024-09-16T14:07:55.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907514+0000", 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.39e-07 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907516+0000", 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4140000000000001e-06 2024-09-16T14:07:55.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2369999999999997e-06 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907531+0000", 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022359680000000001 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909770+0000", 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6520000000000002e-06 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44441 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908453+0000", 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.678814484, 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68963335400000003, 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.187 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44441 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908453+0000", 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908453+0000", 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908453+0000", 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908456+0000", 2024-09-16T14:07:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1169999999999998e-06 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908462+0000", 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0270000000000001e-06 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908470+0000", 2024-09-16T14:07:55.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3180000000000007e-06 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910717+0000", 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022473440000000001 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910720+0000", 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6199999999999999e-06 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44708 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909246+0000", 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67802193799999999, 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68887394000000002, 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44708 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909246+0000", 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909246+0000", 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909246+0000", 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4999999999999998e-07 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909248+0000", 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0150000000000002e-06 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909256+0000", 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8779999999999998e-06 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909263+0000", 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2359999999999998e-06 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911577+0000", 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002313795 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911579+0000", 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3810000000000002e-06 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67734656299999996, 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68823014199999999, 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:07:55.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:07:55.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:43938 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907535+0000", 2024-09-16T14:07:55.201 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.679732, 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.69064857199999996, 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 43938 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907535+0000", 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907535+0000", 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907536+0000", 2024-09-16T14:07:55.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9700000000000002e-07 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907538+0000", 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3769999999999999e-06 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907544+0000", 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8989999999999998e-06 2024-09-16T14:07:55.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6310000000000002e-06 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909798+0000", 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022489379999999998 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44442 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908474+0000", 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67879340499999996, 2024-09-16T14:07:55.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68974340300000003, 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44442 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908474+0000", 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908474+0000", 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908474+0000", 2024-09-16T14:07:55.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908481+0000", 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1890000000000004e-06 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908488+0000", 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8820000000000003e-06 2024-09-16T14:07:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908491+0000", 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6160000000000002e-06 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910742+0000", 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022511240000000002 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910745+0000", 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5050000000000002e-06 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44944 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909268+0000", 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67799986800000001, 2024-09-16T14:07:55.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68898167300000002, 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44944 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909268+0000", 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909268+0000", 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909268+0000", 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909270+0000", 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8479999999999999e-06 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909275+0000", 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:07:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909279+0000", 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.444e-06 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911599+0000", 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023197550000000002 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911601+0000", 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3700000000000002e-06 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.677329811, 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68834439700000005, 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44175 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907753+0000", 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.679514222, 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.69056030000000002, 2024-09-16T14:07:55.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44175 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907753+0000", 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907753+0000", 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907754+0000", 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8000000000000004e-07 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907758+0000", 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2139999999999998e-06 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907769+0000", 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0447e-05 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907775+0000", 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8379999999999998e-06 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020535990000000001 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909831+0000", 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5560000000000002e-06 2024-09-16T14:07:55.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44443 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908495+0000", 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67877249900000003, 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68985148799999996, 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44443 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908495+0000", 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908495+0000", 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908495+0000", 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0100000000000001e-07 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908497+0000", 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4890000000000001e-06 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908500+0000", 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.551e-06 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908503+0000", 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2029999999999999e-06 2024-09-16T14:07:55.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910772+0000", 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022693430000000001 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910774+0000", 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5919999999999999e-06 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44945 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909290+0000", 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67797772999999995, 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68908817700000002, 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44945 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909290+0000", 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909290+0000", 2024-09-16T14:07:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909290+0000", 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909292+0000", 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7439999999999999e-06 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909298+0000", 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8769999999999998e-06 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909301+0000", 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6260000000000001e-06 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911623+0000", 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002321938 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911626+0000", 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4789999999999999e-06 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67731254699999999, 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68845583899999996, 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:07:55.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44176 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907780+0000", 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67948767799999998, 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.69066273499999997, 2024-09-16T14:07:55.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44176 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907780+0000", 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907780+0000", 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907780+0000", 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.37e-07 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907783+0000", 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.435e-06 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907789+0000", 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.156e-06 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907793+0000", 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3429999999999998e-06 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909879+0000", 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020852420000000002 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1110000000000002e-06 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44444 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908506+0000", 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.678761744, 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.689969528, 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44444 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908506+0000", 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908506+0000", 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908506+0000", 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6e-07 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908507+0000", 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2810000000000001e-06 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908516+0000", 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5189999999999992e-06 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908519+0000", 2024-09-16T14:07:55.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5640000000000001e-06 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910797+0000", 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022778410000000001 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910800+0000", 2024-09-16T14:07:55.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5299999999999999e-06 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44946 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909323+0000", 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67794433200000004, 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68918497999999995, 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44946 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909323+0000", 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909323+0000", 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909323+0000", 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.53e-07 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909325+0000", 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.59e-06 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909330+0000", 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8119999999999998e-06 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909332+0000", 2024-09-16T14:07:55.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911646+0000", 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023131580000000001 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911648+0000", 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3460000000000001e-06 2024-09-16T14:07:55.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.677273711, 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68854580200000004, 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:07:55.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:07:55.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44177 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907798+0000", 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67946993899999997, 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.69077543299999999, 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44177 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907798+0000", 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907798+0000", 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907798+0000", 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.16e-07 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907801+0000", 2024-09-16T14:07:55.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907808+0000", 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3150000000000003e-06 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907812+0000", 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0520000000000003e-06 2024-09-16T14:07:55.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021113019999999998 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909926+0000", 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6809999999999998e-06 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.264 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44445 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.908523+0000", 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67874420700000004, 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.69008119300000004, 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44445 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908523+0000", 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908523+0000", 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908524+0000", 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2399999999999999e-07 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908526+0000", 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2120000000000002e-06 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908532+0000", 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0100000000000001e-06 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.908536+0000", 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.126e-06 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910826+0000", 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002290289 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910829+0000", 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5620000000000002e-06 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44947 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909335+0000", 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67793252699999995, 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68930251099999995, 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44947 2024-09-16T14:07:55.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909335+0000", 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909335+0000", 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909335+0000", 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909336+0000", 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2359999999999999e-06 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909340+0000", 2024-09-16T14:07:55.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.349e-06 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909342+0000", 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1909999999999999e-06 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911672+0000", 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023302890000000001 2024-09-16T14:07:55.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.911675+0000", 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.385e-06 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3.5fee67f3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.67723888300000001, 2024-09-16T14:07:55.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.68864061899999995, 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:55.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:07:55.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:07:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:55.279 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 134 2024-09-16T14:07:55.279 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:55.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:07:55.283 INFO:teuthology.orchestra.run.smithi079.stdout:2362 write 0x83f4e8f thru 0x83fae90 (0x6002 bytes) 2024-09-16T14:07:55.283 INFO:teuthology.orchestra.run.smithi079.stdout:2363 write 0x4c0905 thru 0x4cd4f3 (0xcbef bytes) 2024-09-16T14:07:55.283 INFO:teuthology.orchestra.run.smithi079.stdout:2364 write 0x8a1e558 thru 0x8a2d7ef (0xf298 bytes) 2024-09-16T14:07:55.283 INFO:teuthology.orchestra.run.smithi079.stdout:2366 read 0xad4760 thru 0xad7fc0 (0x3861 bytes) 2024-09-16T14:07:55.283 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:55.274+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:55.353 INFO:teuthology.orchestra.run.smithi079.stdout:2367 punch from 0x7c9edbb to 0x7caea6a, (0xfcaf bytes) 2024-09-16T14:07:55.353 INFO:teuthology.orchestra.run.smithi079.stdout:2368 punch from 0xb1131a4 to 0xb11684d, (0x36a9 bytes) 2024-09-16T14:07:55.353 INFO:teuthology.orchestra.run.smithi079.stdout:2369 punch from 0x3733851 to 0x37343cb, (0xb7a bytes) 2024-09-16T14:07:55.353 INFO:teuthology.orchestra.run.smithi079.stdout:2372 writesame 0x9543b33 thru 0x9549f75 data_size 0x1(0x6443 bytes) 2024-09-16T14:07:55.382 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:55.374+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:55.436 INFO:teuthology.orchestra.run.smithi079.stdout:2374 trunc from 0xce5676f to 0xa98a05b 2024-09-16T14:07:55.467 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:55.468 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:55.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:55.482 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:55.482 INFO:teuthology.orchestra.run.smithi079.stdout:2378 read 0x1a5a97f thru 0x1a5f136 (0x47b8 bytes) 2024-09-16T14:07:55.482 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:55.474+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:55.583 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:55.577+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:55.623 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:55.623 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:55.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:55.623 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:55.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.623 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.623 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7097750330000001, 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:07:55.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:07:55.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709765306, 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:07:55.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.634 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7097417429999999, 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.635 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:07:55.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7097145039999999, 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.641 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:07:55.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.646 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709705091, 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:07:55.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:07:55.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709680772, 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.652 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:07:55.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7096611660000001, 2024-09-16T14:07:55.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:07:55.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:07:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.662 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7096409779999999, 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:55.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:07:55.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709617709, 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:07:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:07:55.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709587459, 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.673 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:07:55.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.678 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7095707070000001, 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:55.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:07:55.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.683 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7095534429999999, 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:07:55.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:07:55.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709514607, 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.689 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:55.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:07:55.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709479779, 2024-09-16T14:07:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:07:55.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:07:55.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:55.700 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709443893, 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:55.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:07:55.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:07:55.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7094213009999999, 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:07:55.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7093922290000001, 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.711 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:07:55.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.716 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709373883, 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:55.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:07:55.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.722 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.722 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:55.722 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.709356544, 2024-09-16T14:07:55.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:07:55.722 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.722 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.722 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:55.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:07:55.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:07:55.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.7093406209999999, 2024-09-16T14:07:55.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.732 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:55.733 INFO:teuthology.orchestra.run.smithi079.stdout:2381 read 0x741dd68 thru 0x74220ea (0x4383 bytes) 2024-09-16T14:07:55.733 INFO:teuthology.orchestra.run.smithi079.stdout:2383 trunc from 0xa98a05b to 0x18ffe82 2024-09-16T14:07:55.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:55.735 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:55.735 INFO:teuthology.orchestra.run.smithi079.stdout:2385 trunc from 0x18ffe82 to 0xb23db0a 2024-09-16T14:07:55.735 INFO:teuthology.orchestra.run.smithi079.stdout:2386 read 0xaccbfdd thru 0xacdbbd8 (0xfbfc bytes) 2024-09-16T14:07:55.735 INFO:teuthology.orchestra.run.smithi079.stdout:2387 writesame 0x44973a3 thru 0x449d150 data_size 0x15(0x5dae bytes) 2024-09-16T14:07:55.737 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:55.730+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:55.761 INFO:teuthology.orchestra.run.smithi079.stdout:2388 writesame 0x5b4da56 thru 0x5b4f24f data_size 0x21(0x17fa bytes) 2024-09-16T14:07:55.775 INFO:teuthology.orchestra.run.smithi079.stdout:2389 read 0x8b1f5b6 thru 0x8b1f836 (0x281 bytes) 2024-09-16T14:07:55.775 INFO:teuthology.orchestra.run.smithi079.stdout:2390 punch from 0x4585cfb to 0x4592227, (0xc52c bytes) 2024-09-16T14:07:55.775 INFO:teuthology.orchestra.run.smithi079.stdout:2391 trunc from 0xb23db0a to 0xbceb21c 2024-09-16T14:07:55.799 INFO:teuthology.orchestra.run.smithi079.stdout:2392 write 0xd2ef6c4 thru 0xd2f0193 (0xad0 bytes) 2024-09-16T14:07:55.810 INFO:teuthology.orchestra.run.smithi079.stdout:2393 read 0x758216f thru 0x758527d (0x310f bytes) 2024-09-16T14:07:55.822 INFO:teuthology.orchestra.run.smithi079.stdout:2394 write 0xe4139e0 thru 0xe41be1e (0x843f bytes) 2024-09-16T14:07:55.837 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:55.829+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:55.852 INFO:teuthology.orchestra.run.smithi079.stdout:2395 write 0x13f4dad thru 0x13f9997 (0x4beb bytes) 2024-09-16T14:07:55.854 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:54916 3.1c 3.6dbd6d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:55.853645+0000", 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00033850299999999998, 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035943299999999999, 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 54916 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:55.853645+0000", 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:55.853645+0000", 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:55.853645+0000", 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3399999999999997e-07 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:55.853650+0000", 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5689999999999998e-06 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:55.853672+0000", 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1860000000000001e-05 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:55.853678+0000", 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.896e-06 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-16T14:07:55.858 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:55.866 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:55.938 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:55.930+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:56.039 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:56.034+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:56.044 INFO:teuthology.orchestra.run.smithi079.stdout:2396 write 0x418b12b thru 0x4198cc4 (0xdb9a bytes) 2024-09-16T14:07:56.070 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:56.070 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:56.076 INFO:teuthology.orchestra.run.smithi079.stdout:2397 punch from 0x6333a4b to 0x63361d8, (0x278d bytes) 2024-09-16T14:07:56.076 INFO:teuthology.orchestra.run.smithi079.stdout:2399 write 0x175758e thru 0x175ad70 (0x37e3 bytes) 2024-09-16T14:07:56.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:56.140 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:56.134+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:56.238 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3314402049999998, 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:07:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:07:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3314174969999999, 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:07:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:07:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.331393345, 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:07:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:07:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3313604039999998, 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:07:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:07:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3313357209999999, 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:07:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3313124919999999, 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:07:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3294665280000002, 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:07:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:07:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:07:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3294236640000001, 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.327964046, 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:07:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.327948277, 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:07:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:07:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:07:56.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.327932525, 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.291 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:07:56.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:07:56.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.327917094, 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:07:56.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.301 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3279014560000002, 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:07:56.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:07:56.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.306 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3278863169999999, 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.307 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:07:56.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:07:56.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3278714799999998, 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.312 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:07:56.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:07:56.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3278484759999998, 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3278332509999999, 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:07:56.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:07:56.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3278180669999999, 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:07:56.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:07:56.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3278028219999998, 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:07:56.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:07:56.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:07:56.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:07:56.339 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.3277876640000001, 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:07:56.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.345 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:56.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:56.347 INFO:teuthology.orchestra.run.smithi079.stdout:2400 writesame 0xeac18b0 thru 0xeac3078 data_size 0x1(0x17c9 bytes) 2024-09-16T14:07:56.347 INFO:teuthology.orchestra.run.smithi079.stdout:2401 write 0xdc4d385 thru 0xdc53d38 (0x69b4 bytes) 2024-09-16T14:07:56.347 INFO:teuthology.orchestra.run.smithi079.stdout:2402 trunc from 0xeac3079 to 0x7f2dc15 2024-09-16T14:07:56.347 INFO:teuthology.orchestra.run.smithi079.stdout:2403 read 0x4b46d9e thru 0x4b54d01 (0xdf64 bytes) 2024-09-16T14:07:56.347 INFO:teuthology.orchestra.run.smithi079.stdout:2404 write 0x78ef3aa thru 0x78feec7 (0xfb1e bytes) 2024-09-16T14:07:56.348 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:56.338+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:56.371 INFO:teuthology.orchestra.run.smithi079.stdout:2405 write 0x3c76e2e thru 0x3c783a5 (0x1578 bytes) 2024-09-16T14:07:56.387 INFO:teuthology.orchestra.run.smithi079.stdout:2407 read 0x2c7c92b thru 0x2c7f850 (0x2f26 bytes) 2024-09-16T14:07:56.403 INFO:teuthology.orchestra.run.smithi079.stdout:2408 write 0xc1165e7 thru 0xc11e57f (0x7f99 bytes) 2024-09-16T14:07:56.430 INFO:teuthology.orchestra.run.smithi079.stdout:2411 read 0xa05ecb1 thru 0xa065762 (0x6ab2 bytes) 2024-09-16T14:07:56.430 INFO:teuthology.orchestra.run.smithi079.stdout:2412 read 0xba9a4d thru 0xbb08fc (0x6eb0 bytes) 2024-09-16T14:07:56.441 INFO:teuthology.orchestra.run.smithi079.stdout:2413 write 0xcc6fb0c thru 0xcc75e1e (0x6313 bytes) 2024-09-16T14:07:56.442 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:56.442 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:56.442 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:56.442 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:56.447 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:56.442+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:56.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:56.466 INFO:teuthology.orchestra.run.smithi079.stdout:2414 write 0x6ce82f7 thru 0x6cee9f0 (0x66fa bytes) 2024-09-16T14:07:56.491 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:56.507 INFO:teuthology.orchestra.run.smithi079.stdout:2416 trunc from 0xcc75e1f to 0xfea274 2024-09-16T14:07:56.552 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:56.545+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:56.577 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:56.577 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:56.577 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:56.577 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:56.577 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:56.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:56.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:07:56.581+0000 7f3046625300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T14:07:56.648 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:56.642+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:56.724 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:56.724 INFO:teuthology.orchestra.run.smithi079.stdout:2418 read 0xe1a4a4 thru 0xe1d0a2 (0x2bff bytes) 2024-09-16T14:07:56.749 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:56.742+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:56.761 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:56.761 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62746 3.2 3:4472cc60:::rbd_data.113681563d6c.0000000000000004:15 [read 1287168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.593977+0000", 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.163979182, 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043940676999999997, 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.762 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62746 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593977+0000", 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593977+0000", 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593977+0000", 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593979+0000", 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593985+0000", 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7459999999999997e-06 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593988+0000", 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4640000000000002e-06 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.637801+0000", 2024-09-16T14:07:56.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043812201000000002 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.637864+0000", 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.389e-05 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.637917+0000", 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3032000000000003e-05 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62747 3.2 3:4472cc60:::rbd_data.113681563d6c.0000000000000004:15 [read 1299456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.593992+0000", 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.163963623, 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.051996343, 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.767 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62747 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593992+0000", 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593992+0000", 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593993+0000", 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.593995+0000", 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9599999999999999e-06 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594000+0000", 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594004+0000", 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4350000000000001e-06 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.640959+0000", 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.046955545000000001 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641034+0000", 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5179999999999995e-05 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.645989+0000", 2024-09-16T14:07:56.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049542370000000002 2024-09-16T14:07:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62791 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1053696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.772 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594007+0000", 2024-09-16T14:07:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16394866899999999, 2024-09-16T14:07:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047233243000000001, 2024-09-16T14:07:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62791 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594007+0000", 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594007+0000", 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594008+0000", 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594010+0000", 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.968e-06 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594016+0000", 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7750000000000004e-06 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594020+0000", 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7869999999999999e-06 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641112+0000", 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047092223000000002 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641184+0000", 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1999e-05 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641241+0000", 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6103000000000001e-05 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62792 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1065984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594024+0000", 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16393190199999999, 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047356626999999998, 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62792 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594024+0000", 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594024+0000", 2024-09-16T14:07:56.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594025+0000", 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9699999999999996e-07 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594027+0000", 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9870000000000002e-06 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594033+0000", 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1829999999999999e-06 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594037+0000", 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8029999999999998e-06 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641262+0000", 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047225528000000003 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641328+0000", 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5456999999999993e-05 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641381+0000", 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3072000000000003e-05 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62793 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1078272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594041+0000", 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16391523999999999, 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047482381999999997, 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62793 2024-09-16T14:07:56.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594041+0000", 2024-09-16T14:07:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594041+0000", 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594041+0000", 2024-09-16T14:07:56.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4999999999999998e-07 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594043+0000", 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.742e-06 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594048+0000", 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5300000000000004e-06 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594052+0000", 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4010000000000001e-06 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641400+0000", 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047348068 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641464+0000", 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4478000000000002e-05 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641523+0000", 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8712999999999998e-05 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62794 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1090560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594056+0000", 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16390041399999999, 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047609959, 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62794 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594056+0000", 2024-09-16T14:07:56.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594056+0000", 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594056+0000", 2024-09-16T14:07:56.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9499999999999998e-07 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594058+0000", 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.801e-06 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594063+0000", 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6969999999999998e-06 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594067+0000", 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3969999999999998e-06 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641543+0000", 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047476042000000003 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641608+0000", 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5171000000000003e-05 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641666+0000", 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7456e-05 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62795 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1102848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594071+0000", 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.163885315, 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047735991999999998, 2024-09-16T14:07:56.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62795 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594071+0000", 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594071+0000", 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594071+0000", 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594073+0000", 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7749999999999999e-06 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594078+0000", 2024-09-16T14:07:56.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6450000000000002e-06 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594082+0000", 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3519999999999998e-06 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641685+0000", 2024-09-16T14:07:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047602696999999999 2024-09-16T14:07:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641751+0000", 2024-09-16T14:07:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6730000000000007e-05 2024-09-16T14:07:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641807+0000", 2024-09-16T14:07:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5392000000000001e-05 2024-09-16T14:07:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62796 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1115136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594086+0000", 2024-09-16T14:07:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.163870188, 2024-09-16T14:07:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047859609999999997, 2024-09-16T14:07:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62796 2024-09-16T14:07:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594086+0000", 2024-09-16T14:07:56.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594086+0000", 2024-09-16T14:07:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594086+0000", 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594088+0000", 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.843e-06 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594094+0000", 2024-09-16T14:07:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5910000000000003e-06 2024-09-16T14:07:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594097+0000", 2024-09-16T14:07:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4850000000000001e-06 2024-09-16T14:07:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641826+0000", 2024-09-16T14:07:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047729019999999997 2024-09-16T14:07:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641891+0000", 2024-09-16T14:07:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5378999999999999e-05 2024-09-16T14:07:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641945+0000", 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3919000000000003e-05 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62797 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1127424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594101+0000", 2024-09-16T14:07:56.807 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.163855165, 2024-09-16T14:07:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047979911, 2024-09-16T14:07:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62797 2024-09-16T14:07:56.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594101+0000", 2024-09-16T14:07:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594101+0000", 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594101+0000", 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.53e-07 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594103+0000", 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6619999999999999e-06 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594108+0000", 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.541e-06 2024-09-16T14:07:56.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594112+0000", 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5159999999999999e-06 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.641964+0000", 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047852510000000001 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642026+0000", 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1783000000000001e-05 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642081+0000", 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4546000000000003e-05 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62798 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1139712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594116+0000", 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16383990500000001, 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048100594000000003, 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62798 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594116+0000", 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594116+0000", 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594116+0000", 2024-09-16T14:07:56.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:07:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594118+0000", 2024-09-16T14:07:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.883e-06 2024-09-16T14:07:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594124+0000", 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7599999999999999e-06 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594128+0000", 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5659999999999998e-06 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642100+0000", 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.047972500000000001 2024-09-16T14:07:56.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642162+0000", 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2156000000000006e-05 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642217+0000", 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4355999999999999e-05 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62799 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1152000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594133+0000", 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16382291800000001, 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048220197999999999, 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62799 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594133+0000", 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594133+0000", 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594133+0000", 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594152+0000", 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.88e-05 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594159+0000", 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7730000000000002e-06 2024-09-16T14:07:56.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594163+0000", 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1069999999999998e-06 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642236+0000", 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048072918999999999 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642299+0000", 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2866000000000004e-05 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642353+0000", 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4367000000000003e-05 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62800 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1164288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594168+0000", 2024-09-16T14:07:56.823 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16378833300000001, 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048325311000000003, 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62800 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594168+0000", 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594168+0000", 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594168+0000", 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5799999999999999e-07 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594170+0000", 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0700000000000001e-06 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594176+0000", 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6579999999999999e-06 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594179+0000", 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4000000000000001e-06 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642373+0000", 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048193985000000002 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642439+0000", 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5821000000000006e-05 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642493+0000", 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3718999999999998e-05 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62801 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1176576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594183+0000", 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16377265799999999, 2024-09-16T14:07:56.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048448372000000003, 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62801 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594183+0000", 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594183+0000", 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594184+0000", 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594186+0000", 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:07:56.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594192+0000", 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8649999999999998e-06 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594195+0000", 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5710000000000002e-06 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642513+0000", 2024-09-16T14:07:56.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048317366 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642579+0000", 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6066000000000001e-05 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642632+0000", 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2952999999999999e-05 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62802 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1188864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594199+0000", 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16375663300000001, 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048572096000000002, 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62802 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594199+0000", 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594199+0000", 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594200+0000", 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1300000000000001e-07 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594201+0000", 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.703e-06 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594207+0000", 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5849999999999999e-06 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594211+0000", 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6210000000000001e-06 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642652+0000", 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048441326 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642719+0000", 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6867999999999997e-05 2024-09-16T14:07:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642771+0000", 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2580000000000001e-05 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62803 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1201152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.594215+0000", 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.16374138799999999, 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048696717, 2024-09-16T14:07:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62803 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594215+0000", 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594215+0000", 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594215+0000", 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0200000000000003e-07 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594217+0000", 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8050000000000001e-06 2024-09-16T14:07:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594222+0000", 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6509999999999998e-06 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.594226+0000", 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5930000000000002e-06 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642791+0000", 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048564947999999997 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642857+0000", 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6183e-05 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642911+0000", 2024-09-16T14:07:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4135000000000003e-05 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62804 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1213440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.597166+0000", 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.160789879, 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.045881528999999997, 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62804 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.597166+0000", 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.597166+0000", 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.597167+0000", 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6400000000000001e-07 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.597177+0000", 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0010000000000001e-05 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598849+0000", 2024-09-16T14:07:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016722410000000001 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598857+0000", 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9359999999999999e-06 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642931+0000", 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044073405000000003 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.642996+0000", 2024-09-16T14:07:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5011999999999994e-05 2024-09-16T14:07:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643048+0000", 2024-09-16T14:07:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2061000000000002e-05 2024-09-16T14:07:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62805 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1225728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.598864+0000", 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.15909153100000001, 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044322931000000003, 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62805 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598864+0000", 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598864+0000", 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598865+0000", 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0900000000000001e-07 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598869+0000", 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5219999999999999e-06 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598877+0000", 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1850000000000004e-06 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598881+0000", 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.233e-06 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643066+0000", 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044185149999999999 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643133+0000", 2024-09-16T14:07:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7102999999999998e-05 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643187+0000", 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4029000000000001e-05 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62806 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1238016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.598886+0000", 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.15907043700000001, 2024-09-16T14:07:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044443857000000003, 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62806 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598886+0000", 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598886+0000", 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598886+0000", 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598888+0000", 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.092e-06 2024-09-16T14:07:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598894+0000", 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.207e-06 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598898+0000", 2024-09-16T14:07:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9140000000000001e-06 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643207+0000", 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044309137999999998 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643276+0000", 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8391000000000003e-05 2024-09-16T14:07:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643329+0000", 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3677999999999997e-05 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62807 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1250304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.598902+0000", 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.15905362300000001, 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044565199, 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62807 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598902+0000", 2024-09-16T14:07:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598902+0000", 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.598903+0000", 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.599050+0000", 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014712899999999999 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.599057+0000", 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1180000000000003e-06 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.599062+0000", 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.797e-06 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643349+0000", 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044287435999999999 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643413+0000", 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3817000000000002e-05 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643468+0000", 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4487000000000002e-05 2024-09-16T14:07:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:62808 3.2 3:47bb3812:::rbd_data.113681563d6c.0000000000000007:15 [read 1262592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:56.599068+0000", 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.158888117, 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.044535714999999997, 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 62808 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.599068+0000", 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.599068+0000", 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.599068+0000", 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0399999999999996e-07 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.599071+0000", 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6740000000000001e-06 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.601534+0000", 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024627799999999999 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.601542+0000", 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.422e-06 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643486+0000", 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.041943606000000001 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643551+0000", 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4634999999999994e-05 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:56.643604+0000", 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2994000000000001e-05 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:56.868 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:56.868 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:56.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:56.870 INFO:teuthology.orchestra.run.smithi079.stdout:2419 punch from 0x73402c to 0x73bf4e, (0x7f22 bytes) 2024-09-16T14:07:56.871 INFO:teuthology.orchestra.run.smithi079.stdout:2421 punch from 0xba9d5c to 0xbb53b6, (0xb65a bytes) 2024-09-16T14:07:56.871 INFO:teuthology.orchestra.run.smithi079.stdout:2422 clone 19 order 18 su 32768 sc 14 2024-09-16T14:07:56.871 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:56.861+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:56.970 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:56.965+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:56.984 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:56.984 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:56.994 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:07:57.071 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:57.066+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:57.083 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone18 from 0xfea274 (overlap 0xfea274) to 0x1976a5 2024-09-16T14:07:57.107 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:57.107 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:57.118 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:07:57.172 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:57.166+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:57.218 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:57.219 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:57.228 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:07:57.272 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:57.265+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:57.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:07:57.372 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:57.365+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:57.382 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:57.382 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:57.394 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:07:57.474 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:07:57.469+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:07:57.504 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:57.504 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:57.514 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:07:57.575 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:57.570+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:57.611 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:57.612 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:57.620 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:07:57.676 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:57.670+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:57.720 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:57.720 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:57.729 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:07:57.768 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-16T14:07:57.776 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:07:57.770+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:07:57.790 INFO:teuthology.orchestra.run.smithi079.stdout:2423 read 0xba4801 thru 0xbaaadb (0x62db bytes) 2024-09-16T14:07:57.807 INFO:teuthology.orchestra.run.smithi079.stdout:2424 trunc from 0xfea274 to 0x4f9054b 2024-09-16T14:07:57.817 INFO:teuthology.orchestra.run.smithi079.stdout:2425 write 0xf136a thru 0xf5a9f (0x4736 bytes) 2024-09-16T14:07:57.828 INFO:teuthology.orchestra.run.smithi079.stdout:2426 write 0x1d94eca thru 0x1d9f6b8 (0xa7ef bytes) 2024-09-16T14:07:57.830 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:57.830 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:57.832 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:57.832 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:57.832 INFO:teuthology.orchestra.run.smithi079.stdout:2430 punch from 0x5ccdc6 to 0x5d59dd, (0x8c17 bytes) 2024-09-16T14:07:57.832 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:57.839 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:07:57.844 INFO:teuthology.orchestra.run.smithi079.stdout:2432 write 0x91c6222 thru 0x91d169e (0xb47d bytes) 2024-09-16T14:07:57.850 INFO:teuthology.orchestra.run.smithi079.stdout:2433 write 0xb720be thru 0xb797da (0x771d bytes) 2024-09-16T14:07:57.863 INFO:teuthology.orchestra.run.smithi079.stdout:2434 punch from 0x90dcdba to 0x90e56d8, (0x891e bytes) 2024-09-16T14:07:57.869 INFO:teuthology.orchestra.run.smithi079.stdout:2435 trunc from 0x91d169f to 0xd7e9efe 2024-09-16T14:07:57.875 INFO:teuthology.orchestra.run.smithi079.stdout:2436 write 0x9a39222 thru 0x9a39d56 (0xb35 bytes) 2024-09-16T14:07:57.877 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:57.870+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:57.878 INFO:teuthology.orchestra.run.smithi079.stdout:2437 write 0x3e35fa9 thru 0x3e434a4 (0xd4fc bytes) 2024-09-16T14:07:57.884 INFO:teuthology.orchestra.run.smithi079.stdout:2440 write 0x4fc1602 thru 0x4fccbf2 (0xb5f1 bytes) 2024-09-16T14:07:57.889 INFO:teuthology.orchestra.run.smithi079.stdout:2441 clone 20 order 19 su 2048 sc 10 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:3.d 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:3.1d 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:3.17 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:3.16 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:3.13 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:3.9 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:3.12 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:3.11 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:2.7 2024-09-16T14:07:57.906 INFO:teuthology.orchestra.run.smithi079.stdout:2.5 2024-09-16T14:07:57.907 INFO:teuthology.orchestra.run.smithi079.stdout:2.3 2024-09-16T14:07:57.907 INFO:teuthology.orchestra.run.smithi079.stdout:2.2 2024-09-16T14:07:57.907 INFO:teuthology.orchestra.run.smithi079.stdout:3.a 2024-09-16T14:07:57.907 INFO:teuthology.orchestra.run.smithi079.stdout:3.14 2024-09-16T14:07:57.907 INFO:teuthology.orchestra.run.smithi079.stdout:2.0 2024-09-16T14:07:57.935 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:57.935 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:57.944 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:07:57.961 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:07:57.978 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:57.970+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:58.042 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:58.042 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:58.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:07:58.095 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:58.090+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:58.163 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone19 from 0xd7e9efe (overlap 0xfea274) to 0x32ed4a 2024-09-16T14:07:58.180 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:58.174+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:58.210 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:58.211 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:58.220 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:07:58.281 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:58.274+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:58.312 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:07:58.338 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:07:58.338 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:07:58.348 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:07:58.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:07:58.383 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:07:58.391 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:58.386+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:58.409 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:07:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:07:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:07:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:07:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:07:58.409 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:07:58.410 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:07:58.410 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:07:58.410 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:07:58.413 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:07:58.413 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:07:58.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 2.5 --file /var/log/ceph/exp.2.5.3 2024-09-16T14:07:58.483 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:58.477+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:58.492 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:58.492 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:58.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:81472 3.c 3:30787fdc:::rbd_data.113618577e70.0000000000000242:head [delete] snapc 17=[17] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:58.486838+0000", 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0043771449999999998, 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043980800000000004, 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 81472 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.486838+0000", 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.486838+0000", 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.486839+0000", 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9400000000000004e-07 2024-09-16T14:07:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.486843+0000", 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6319999999999999e-06 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.486853+0000", 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4839999999999993e-06 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.486859+0000", 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4629999999999998e-06 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.486922+0000", 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2632999999999995e-05 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.486996+0000", 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3765999999999998e-05 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 7", 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.488319+0000", 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001323142 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:58.489444+0000", 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011256759999999999 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:07:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-16T14:07:58.498 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:58.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:07:58.584 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:07:58.577+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:07:58.610 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:58.610 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:58.610 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:58.610 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:58.610 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:58.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:07:58.684 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:58.677+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:58.772 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8582560170000004, 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:07:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:07:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:07:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:07:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:07:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:07:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8582462900000003, 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:07:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:07:58.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8582227270000002, 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:07:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:07:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8581954879999998, 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:07:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:07:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:07:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8581860749999999, 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8581617560000003, 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:07:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8581421499999999, 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:07:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8581219620000002, 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:07:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:07:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8580986929999996, 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:07:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8580684429999996, 2024-09-16T14:07:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:07:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.858051691, 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8580344269999998, 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:07:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:07:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8579955909999999, 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:07:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:07:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8579607630000003, 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:07:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:07:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8579248770000003, 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:07:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8579022849999998, 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:07:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8578732129999995, 2024-09-16T14:07:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:07:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8578548670000004, 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:07:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8578375280000001, 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:07:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:07:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.8578216049999998, 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:07:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:58.878 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:58.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:07:58.880 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-16T14:07:58.881 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:58.881 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:58.881 INFO:teuthology.orchestra.run.smithi079.stdout:2446 read 0x5e4b635 thru 0x5e4dbae (0x257a bytes) 2024-09-16T14:07:58.881 INFO:teuthology.orchestra.run.smithi079.stdout:2447 read 0x87e6fd4 thru 0x87ec139 (0x5166 bytes) 2024-09-16T14:07:58.881 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:58.874+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:58.900 INFO:teuthology.orchestra.run.smithi079.stdout:2448 trunc from 0xd7e9efe to 0x6f6c474 2024-09-16T14:07:58.954 INFO:teuthology.orchestra.run.smithi079.stdout:2449 writesame 0x45cea25 thru 0x45de310 data_size 0x59(0xf8ec bytes) 2024-09-16T14:07:58.979 INFO:teuthology.orchestra.run.smithi079.stdout:2450 punch from 0x1f83dca to 0x1f92680, (0xe8b6 bytes) 2024-09-16T14:07:58.980 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:58.973+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:58.990 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:58.990 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:58.990 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:58.991 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:58.993 INFO:teuthology.orchestra.run.smithi079.stdout:2454 writesame 0x9bacada thru 0x9bb20d4 data_size 0x45(0x55fb bytes) 2024-09-16T14:07:58.993 INFO:teuthology.orchestra.run.smithi079.stdout:2456 trunc from 0x9bb20d5 to 0x99ab1d7 2024-09-16T14:07:58.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:07:59.011 INFO:teuthology.orchestra.run.smithi079.stdout:2458 trunc from 0x99ab1d7 to 0xe0f4f78 2024-09-16T14:07:59.025 INFO:teuthology.orchestra.run.smithi079.stdout:2459 write 0x114e473 thru 0x1151144 (0x2cd2 bytes) 2024-09-16T14:07:59.077 INFO:teuthology.orchestra.run.smithi079.stdout:2461 write 0x235a8d9 thru 0x236a8c5 (0xffed bytes) 2024-09-16T14:07:59.081 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:59.073+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:59.097 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:59.097 INFO:teuthology.orchestra.run.smithi079.stdout:2463 write 0x974b4ac thru 0x974bb32 (0x687 bytes) 2024-09-16T14:07:59.105 INFO:teuthology.orchestra.run.smithi079.stdout:2464 read 0x5815a77 thru 0x5823c9b (0xe225 bytes) 2024-09-16T14:07:59.105 INFO:teuthology.orchestra.run.smithi079.stdout:2465 read 0xc137254 thru 0xc13eb28 (0x78d5 bytes) 2024-09-16T14:07:59.107 INFO:teuthology.orchestra.run.smithi079.stdout:2466 read 0xca9bc1 thru 0xcaee18 (0x5258 bytes) 2024-09-16T14:07:59.127 INFO:teuthology.orchestra.run.smithi079.stdout:2467 punch from 0x32536b2 to 0x3256349, (0x2c97 bytes) 2024-09-16T14:07:59.135 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:59.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:59.136 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:59.136 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:59.136 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:59.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:07:59.191 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:59.185+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:59.237 INFO:teuthology.orchestra.run.smithi079.stdout:2468 read 0x84047a0 thru 0x84111cd (0xca2e bytes) 2024-09-16T14:07:59.239 INFO:teuthology.orchestra.run.smithi079.stdout:2469 read 0x37ff2e6 thru 0x380b611 (0xc32c bytes) 2024-09-16T14:07:59.253 INFO:teuthology.orchestra.run.smithi079.stdout:2470 trunc from 0xe0f4f78 to 0xad78e38 2024-09-16T14:07:59.283 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:59.278+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:59.319 INFO:teuthology.orchestra.run.smithi079.stdout:2472 read 0x8840da3 thru 0x884fcd8 (0xef36 bytes) 2024-09-16T14:07:59.323 INFO:teuthology.orchestra.run.smithi079.stdout:2473 trunc from 0xad78e38 to 0xd5f79ac 2024-09-16T14:07:59.326 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:59.326 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:59.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:59.326 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:59.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.326 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.326 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4194328000000001, 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:07:59.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:07:59.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4194100919999997, 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.332 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:07:59.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:07:59.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:07:59.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:07:59.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4193859399999997, 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:07:59.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:07:59.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.419352999, 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:07:59.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:07:59.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4193283159999996, 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.348 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:07:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4193050869999997, 2024-09-16T14:07:59.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:07:59.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:07:59.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4174591230000004, 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:07:59.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:07:59.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4174162590000003, 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:07:59.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4159566410000002, 2024-09-16T14:07:59.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:07:59.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:07:59.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:07:59.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:07:59.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4159408720000002, 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:07:59.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:07:59.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4159251199999998, 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:07:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:07:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:07:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4159096890000002, 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:07:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:07:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4158940510000004, 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:07:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:07:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4158789120000002, 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:07:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:07:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:07:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.415864075, 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.415841071, 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4158258459999997, 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:07:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:07:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4158106620000002, 2024-09-16T14:07:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:07:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:07:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:07:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:07:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:07:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:07:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:07:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:07:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:07:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:07:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:07:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:07:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.415795417, 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:07:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:07:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:07:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:07:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.4157802589999999, 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:07:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:07:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:07:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.442 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:59.443 INFO:teuthology.orchestra.run.smithi079.stdout:2477 read 0xb1f8906 thru 0xb204621 (0xbd1c bytes) 2024-09-16T14:07:59.443 INFO:teuthology.orchestra.run.smithi079.stdout:2478 writesame 0x35efb29 thru 0x35f96a9 data_size 0x4d(0x9b81 bytes) 2024-09-16T14:07:59.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:07:59.445 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:59.445 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:59.446 INFO:teuthology.orchestra.run.smithi079.stdout:2482 read 0x6ca524e thru 0x6cb3a33 (0xe7e6 bytes) 2024-09-16T14:07:59.446 INFO:teuthology.orchestra.run.smithi079.stdout:2483 write 0x998c571 thru 0x998e420 (0x1eb0 bytes) 2024-09-16T14:07:59.446 INFO:teuthology.orchestra.run.smithi079.stdout:2484 read 0x66ba945 thru 0x66c93ed (0xeaa9 bytes) 2024-09-16T14:07:59.446 INFO:teuthology.orchestra.run.smithi079.stdout:2485 trunc from 0xd5f79ac to 0x44ab68d 2024-09-16T14:07:59.446 INFO:teuthology.orchestra.run.smithi079.stdout:2486 punch from 0x43c6aec to 0x43ce2c5, (0x77d9 bytes) 2024-09-16T14:07:59.446 INFO:teuthology.orchestra.run.smithi079.stdout:2487 read 0x2dbc8f4 thru 0x2dbd6e4 (0xdf1 bytes) 2024-09-16T14:07:59.446 INFO:teuthology.orchestra.run.smithi079.stdout:2490 writesame 0xe69fcd thru 0xe6f12a data_size 0x5(0x515e bytes) 2024-09-16T14:07:59.446 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:59.438+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:59.457 INFO:teuthology.orchestra.run.smithi079.stdout:2491 write 0x5ba5090 thru 0x5bb3af4 (0xea65 bytes) 2024-09-16T14:07:59.480 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:59.480 INFO:teuthology.orchestra.run.smithi079.stdout:2493 punch from 0x1acbad9 to 0x1ad31d5, (0x76fc bytes) 2024-09-16T14:07:59.480 INFO:teuthology.orchestra.run.smithi079.stdout:2494 read 0x20f5e8d thru 0x2100cc4 (0xae38 bytes) 2024-09-16T14:07:59.485 INFO:teuthology.orchestra.run.smithi079.stdout:2495 read 0xc71740 thru 0xc7ee9e (0xd75f bytes) 2024-09-16T14:07:59.490 INFO:teuthology.orchestra.run.smithi079.stdout:2498 write 0x24f893b thru 0x250665d (0xdd23 bytes) 2024-09-16T14:07:59.524 INFO:teuthology.orchestra.run.smithi079.stdout:2499 write 0x884ab67 thru 0x8856776 (0xbc10 bytes) 2024-09-16T14:07:59.544 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:07:59.538+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:07:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:2500 read 0x5c07cb9 thru 0x5c0a8d1 (0x2c19 bytes) 2024-09-16T14:07:59.547 INFO:teuthology.orchestra.run.smithi079.stdout:2501 write 0x43d296e thru 0x43d7e63 (0x54f6 bytes) 2024-09-16T14:07:59.550 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:07:59.552 INFO:teuthology.orchestra.run.smithi079.stdout:2504 trunc from 0x8856777 to 0x936922e 2024-09-16T14:07:59.556 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:59.556 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:59.556 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:07:59.556 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:59.561 INFO:teuthology.orchestra.run.smithi079.stdout:2506 punch from 0x521fe07 to 0x52221a7, (0x23a0 bytes) 2024-09-16T14:07:59.561 INFO:teuthology.orchestra.run.smithi079.stdout:2508 trunc from 0x936922e to 0x920f378 2024-09-16T14:07:59.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:07:59.571 INFO:teuthology.orchestra.run.smithi079.stdout:2510 punch from 0x67b00dc to 0x67b5f70, (0x5e94 bytes) 2024-09-16T14:07:59.571 INFO:teuthology.orchestra.run.smithi079.stdout:2512 read 0x67855cf thru 0x6787aed (0x251f bytes) 2024-09-16T14:07:59.571 INFO:teuthology.orchestra.run.smithi079.stdout:2513 clone 21 order 25 su 16777216 sc 14 2024-09-16T14:07:59.645 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:59.638+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:59.647 INFO:teuthology.orchestra.run.smithi079.stderr:Exporting 2.5 info 2.5( empty local-lis/les=19/20 n=0 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-16T14:07:59.650 INFO:teuthology.orchestra.run.smithi079.stderr:Export successful 2024-09-16T14:07:59.668 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:59.668 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:07:59.668 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:07:59.668 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:07:59.669 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:59.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:07:59.746 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:07:59.742+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:07:59.777 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:07:59.777 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86627 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 115712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.60145610900000002, 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057990535000000003, 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86627 2024-09-16T14:07:59.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.79e-07 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172073+0000", 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.53e-06 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172077+0000", 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172079+0000", 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224325+0000", 2024-09-16T14:07:59.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052246133 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224394+0000", 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8960000000000004e-05 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.230062+0000", 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056682820000000002 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86989 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:07:59.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.579847263, 2024-09-16T14:07:59.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058737710999999998, 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86989 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193681+0000", 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193682+0000", 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6169999999999999e-06 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193687+0000", 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7210000000000002e-06 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193690+0000", 2024-09-16T14:07:59.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.072e-06 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.246881+0000", 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.053191117000000003 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252309+0000", 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054275979999999996 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252418+0000", 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109255 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86995 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57982106099999997, 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059640662999999997, 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86995 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197498+0000", 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037915219999999999 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197504+0000", 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7520000000000002e-06 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197515+0000", 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.117e-05 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197521+0000", 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.083e-06 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253121+0000", 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055600254000000002 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253227+0000", 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010607000000000001 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253347+0000", 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119812 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87017 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.575885325, 2024-09-16T14:07:59.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676182999999998, 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87017 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197643+0000", 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197645+0000", 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197652+0000", 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8430000000000004e-06 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197656+0000", 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254046+0000", 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056390033999999999 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254205+0000", 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158639 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254318+0000", 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113916 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87019 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57586774900000004, 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056580459, 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.797 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87019 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197663+0000", 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.396e-06 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197669+0000", 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4099999999999996e-06 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197673+0000", 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254066+0000", 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056392870999999997 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254160+0000", 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4574000000000006e-05 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254240+0000", 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0125999999999998e-05 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87023 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 193536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57585088200000001, 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056736048999999997, 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87023 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197679+0000", 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197686+0000", 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197690+0000", 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.177e-06 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254263+0000", 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056573364000000001 2024-09-16T14:07:59.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254334+0000", 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0574000000000006e-05 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254413+0000", 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079999999999995e-05 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87027 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.575833759, 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056974745, 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87027 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2599999999999998e-07 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197696+0000", 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1679999999999998e-06 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197703+0000", 2024-09-16T14:07:59.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4389999999999997e-06 2024-09-16T14:07:59.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197707+0000", 2024-09-16T14:07:59.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:07:59.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254383+0000", 2024-09-16T14:07:59.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676509999999999 2024-09-16T14:07:59.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254483+0000", 2024-09-16T14:07:59.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9692999999999996e-05 2024-09-16T14:07:59.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254669+0000", 2024-09-16T14:07:59.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018580300000000001 2024-09-16T14:07:59.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.811 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87029 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 197632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.812 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:07:59.812 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57581692200000001, 2024-09-16T14:07:59.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057332283999999997, 2024-09-16T14:07:59.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87029 2024-09-16T14:07:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:07:59.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197713+0000", 2024-09-16T14:07:59.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.199e-06 2024-09-16T14:07:59.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197719+0000", 2024-09-16T14:07:59.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.699e-06 2024-09-16T14:07:59.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197723+0000", 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000002e-06 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254709+0000", 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056986536999999997 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254790+0000", 2024-09-16T14:07:59.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0192999999999996e-05 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255043+0000", 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025339500000000001 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87031 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 234496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57580063000000004, 2024-09-16T14:07:59.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057120535, 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87031 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0600000000000001e-07 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197729+0000", 2024-09-16T14:07:59.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0499999999999999e-06 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197736+0000", 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2249999999999997e-06 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197739+0000", 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6839999999999998e-06 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254697+0000", 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056957902999999997 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254776+0000", 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254848+0000", 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1044999999999995e-05 2024-09-16T14:07:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87035 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 198656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57578462600000002, 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057340290000000002, 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87035 2024-09-16T14:07:59.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197744+0000", 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6499999999999999e-07 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197746+0000", 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0169999999999999e-06 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197752+0000", 2024-09-16T14:07:59.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1040000000000003e-06 2024-09-16T14:07:59.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197755+0000", 2024-09-16T14:07:59.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.782e-06 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254869+0000", 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057113766000000003 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254971+0000", 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101932 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255083+0000", 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112124 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87039 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 238592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.828 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57576821600000005, 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057586228000000003, 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87039 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197760+0000", 2024-09-16T14:07:59.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197762+0000", 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.829e-06 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197768+0000", 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5470000000000002e-06 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197772+0000", 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4699999999999998e-06 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255109+0000", 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057337232000000002 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255184+0000", 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5593000000000006e-05 2024-09-16T14:07:59.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255346+0000", 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016120899999999999 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87041 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 202752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57575216200000001, 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057827176000000001, 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87041 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197776+0000", 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:07:59.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197778+0000", 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3709999999999998e-06 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197784+0000", 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.054e-06 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197787+0000", 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1879999999999998e-06 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255397+0000", 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057609059999999997 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255483+0000", 2024-09-16T14:07:59.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6415999999999994e-05 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255603+0000", 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119703 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87043 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 203776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57573599499999994, 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057723202000000001, 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87043 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197801+0000", 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5869999999999993e-06 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197807+0000", 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197811+0000", 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0099999999999997e-06 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255377+0000", 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057566063000000001 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255450+0000", 2024-09-16T14:07:59.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3192e-05 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255515+0000", 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4543999999999997e-05 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87045 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57571180200000005, 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057864405000000001, 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87045 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.844 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197819+0000", 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.311e-06 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197824+0000", 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6980000000000003e-06 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197828+0000", 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2339999999999999e-06 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255537+0000", 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057709979000000002 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255613+0000", 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5807000000000001e-05 2024-09-16T14:07:59.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255680+0000", 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6935999999999997e-05 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87051 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57569665000000003, 2024-09-16T14:07:59.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058010645999999999, 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87051 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:07:59.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200527+0000", 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026964990000000002 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200532+0000", 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8899999999999998e-06 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200541+0000", 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1249999999999993e-06 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200546+0000", 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4129999999999997e-06 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255630+0000", 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055083612999999997 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255711+0000", 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1934000000000005e-05 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255842+0000", 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013017200000000001 2024-09-16T14:07:59.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87559 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 100352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57286229399999999, 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056509688000000002, 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87559 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200666+0000", 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200668+0000", 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.21e-06 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200674+0000", 2024-09-16T14:07:59.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.896e-06 2024-09-16T14:07:59.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200678+0000", 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7050000000000001e-06 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.256898+0000", 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056220883999999999 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257014+0000", 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011530499999999999 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257175+0000", 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016127200000000001 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87568 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.57283031500000003, 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056739408999999998, 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87568 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200698+0000", 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200700+0000", 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.272e-06 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200706+0000", 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7080000000000002e-06 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200709+0000", 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.394e-06 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257221+0000", 2024-09-16T14:07:59.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056512383999999999 2024-09-16T14:07:59.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257319+0000", 2024-09-16T14:07:59.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7510000000000007e-05 2024-09-16T14:07:59.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257437+0000", 2024-09-16T14:07:59.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011773399999999999 2024-09-16T14:07:59.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87570 3.e 3:73b2712a:::rbd_data.113619b3ee73.0000000000000027:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.572814983, 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056717201000000002, 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87570 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201471+0000", 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075828899999999999 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201480+0000", 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.448e-06 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201494+0000", 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3162999999999999e-05 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201500+0000", 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0090000000000004e-06 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257205+0000", 2024-09-16T14:07:59.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055705603999999999 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257330+0000", 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124972 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257430+0000", 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9716000000000005e-05 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87651 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 136192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.567915277, 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056431633000000002, 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87651 2024-09-16T14:07:59.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205613+0000", 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999998e-07 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205614+0000", 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205619+0000", 2024-09-16T14:07:59.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.758e-06 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205622+0000", 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0690000000000002e-06 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.261887+0000", 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056265241000000001 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.261965+0000", 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7671999999999998e-05 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262044+0000", 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9320999999999995e-05 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87773 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 63488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.56703796900000003, 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056458201, 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87773 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:07:59.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:07:59.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:07:59.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:07:59.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207001+0000", 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051104000000000004 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207006+0000", 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207019+0000", 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2266000000000001e-05 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207025+0000", 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2029999999999997e-06 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262711+0000", 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055686539 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262801+0000", 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9895999999999994e-05 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262948+0000", 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146642 2024-09-16T14:07:59.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:07:59.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:07:59.880 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:07:59.881 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:07:59.882 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:07:59.877+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:07:59.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:07:59.982 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:07:59.974+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:07:59.994 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:07:59.994 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:07:59.994 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:07:59.995 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:00.005 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:00.083 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:00.077+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:00.105 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:00.105 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:00.106 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:00.115 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:00.184 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:00.177+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:00.201 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone20 from 0x920f378 (overlap 0x44ab68d) to 0x1b77c38 2024-09-16T14:08:00.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 2.5 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1143 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.885570594000001, 2024-09-16T14:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018939362000000001, 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1143 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.44e-07 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336884+0000", 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336891+0000", 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336895+0000", 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0820000000000001e-06 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349584+0000", 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012689283000000001 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349708+0000", 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123793 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350151+0000", 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044259900000000001 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352993+0000", 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028426979999999998 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355720+0000", 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002726642 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:00.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355762+0000", 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2116000000000001e-05 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355820+0000", 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8118999999999997e-05 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.885552488000002, 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.885530091, 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.884923380000004, 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.884849053000003, 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.913924260999998, 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.28596568, 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.755805213999999, 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.108256683, 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84209 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 120832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.115901+0000", 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1065503809999999, 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018052364000000001, 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84209 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.115901+0000", 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.115901+0000", 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.115902+0000", 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.24e-06 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.115908+0000", 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.161e-06 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.115922+0000", 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3675999999999999e-05 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.115929+0000", 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9040000000000003e-06 2024-09-16T14:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.133679+0000", 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017749609 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.133782+0000", 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103154 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.133953+0000", 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017161999999999999 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84213 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 145408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.116197+0000", 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.106253929, 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018049078999999999, 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84213 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116197+0000", 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116197+0000", 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116199+0000", 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4079999999999999e-06 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116205+0000", 2024-09-16T14:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6740000000000002e-06 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116218+0000", 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3022e-05 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116225+0000", 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.917e-06 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.133984+0000", 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017759515999999999 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134079+0000", 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5085000000000002e-05 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134247+0000", 2024-09-16T14:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016745700000000001 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84214 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 169984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.116232+0000", 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1062194329999999, 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01825798, 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84214 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116232+0000", 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116232+0000", 2024-09-16T14:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116233+0000", 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5300000000000004e-07 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116236+0000", 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1350000000000001e-06 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116244+0000", 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3040000000000005e-06 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116249+0000", 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3929999999999999e-06 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134279+0000", 2024-09-16T14:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018029301000000001 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134367+0000", 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8325000000000006e-05 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134490+0000", 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012286900000000001 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84215 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 190464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.116260+0000", 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.106191031, 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018474918999999999, 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84215 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116260+0000", 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116260+0000", 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116261+0000", 2024-09-16T14:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4000000000000002e-07 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116266+0000", 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8770000000000001e-06 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116274+0000", 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3440000000000001e-06 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116279+0000", 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2700000000000004e-06 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134520+0000", 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018240379000000001 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134615+0000", 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5047e-05 2024-09-16T14:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134735+0000", 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120462 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84216 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 215040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.116285+0000", 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1061669199999999, 2024-09-16T14:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018582227, 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84216 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116285+0000", 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116285+0000", 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116285+0000", 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116288+0000", 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116295+0000", 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5440000000000001e-06 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116300+0000", 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5360000000000003e-06 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134764+0000", 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0184637 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134808+0000", 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4042000000000003e-05 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134867+0000", 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9120999999999999e-05 2024-09-16T14:08:00.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84217 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.116305+0000", 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1061466630000001, 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0186968, 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.303 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84217 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116305+0000", 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116305+0000", 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116305+0000", 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999999999999997e-07 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116308+0000", 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6079999999999998e-06 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116315+0000", 2024-09-16T14:08:00.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6260000000000003e-06 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116320+0000", 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5129999999999998e-06 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134889+0000", 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018568850000000001 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.134940+0000", 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1026999999999998e-05 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.135002+0000", 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1686000000000005e-05 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84218 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 260096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.116325+0000", 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1061265769999999, 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018893842000000001, 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84218 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116325+0000", 2024-09-16T14:08:00.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116325+0000", 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116325+0000", 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5999999999999999e-07 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116328+0000", 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.582e-06 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116335+0000", 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5159999999999997e-06 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.116340+0000", 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7140000000000001e-06 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.135024+0000", 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01868365 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.135112+0000", 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7855999999999994e-05 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.135219+0000", 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107064 2024-09-16T14:08:00.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84364 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 204800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.118580+0000", 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.103871252, 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018045302999999999, 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.312 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84364 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118580+0000", 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118580+0000", 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118581+0000", 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.37e-07 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118583+0000", 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118589+0000", 2024-09-16T14:08:00.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2870000000000001e-06 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118593+0000", 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8210000000000003e-06 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136524+0000", 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017930514000000002 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136567+0000", 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3430999999999998e-05 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136625+0000", 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8270999999999998e-05 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84365 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.118598+0000", 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1038537850000001, 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018173272000000001, 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84365 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118598+0000", 2024-09-16T14:08:00.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118598+0000", 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118598+0000", 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5199999999999998e-07 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118600+0000", 2024-09-16T14:08:00.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0810000000000001e-06 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118606+0000", 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118612+0000", 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6729999999999997e-06 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136648+0000", 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018036262000000001 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136700+0000", 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1928000000000002e-05 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136771+0000", 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0772000000000006e-05 2024-09-16T14:08:00.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84366 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.118617+0000", 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1038348389999999, 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018298841999999999, 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84366 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118617+0000", 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118617+0000", 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118617+0000", 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7e-07 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118619+0000", 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3010000000000001e-06 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118626+0000", 2024-09-16T14:08:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5270000000000004e-06 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118630+0000", 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3420000000000001e-06 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136798+0000", 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018168340000000002 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136853+0000", 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5021000000000001e-05 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136915+0000", 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1940999999999995e-05 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:84367 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 249856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.118634+0000", 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.103816962, 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018526974000000002, 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 84367 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118634+0000", 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118634+0000", 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118635+0000", 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118643+0000", 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7579999999999992e-06 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118649+0000", 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3910000000000003e-06 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.118655+0000", 2024-09-16T14:08:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1240000000000002e-06 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.136941+0000", 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018285454999999999 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137003+0000", 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2252e-05 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137161+0000", 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015860399999999999 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.332 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:00.334 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:00.334 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:00.326+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:00.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:00.433 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:00.433 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:00.433 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:00.433 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:00.434 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:00.429+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:00.443 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:00.535 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:00.529+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:00.541 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:00.541 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:00.541 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:00.541 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:00.541 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:00.550 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:00.558 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:00.581 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:00.601 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:00.625 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-16T14:08:00.632 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:00.637 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:00.630+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:00.658 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:00.658 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86470 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 258048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173921+0000", 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.481670788, 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043647279999999997, 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.659 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86470 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173921+0000", 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173921+0000", 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173922+0000", 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173924+0000", 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.086e-06 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173933+0000", 2024-09-16T14:08:00.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3519999999999996e-06 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173938+0000", 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5310000000000003e-06 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217431+0000", 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043493433999999997 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217520+0000", 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9172999999999995e-05 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217569+0000", 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8276000000000003e-05 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86482 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 145408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173943+0000", 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.481649328, 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043821633999999998, 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86482 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173943+0000", 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173943+0000", 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173943+0000", 2024-09-16T14:08:00.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173946+0000", 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2220000000000001e-06 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173955+0000", 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4970000000000007e-06 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173963+0000", 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1540000000000002e-06 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217604+0000", 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043641081999999998 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217692+0000", 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8109000000000006e-05 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217765+0000", 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2127999999999998e-05 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86493 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 222208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173967+0000", 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4816247570000001, 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043945154, 2024-09-16T14:08:00.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86493 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173967+0000", 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173967+0000", 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173968+0000", 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5199999999999998e-07 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173970+0000", 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8309999999999999e-06 2024-09-16T14:08:00.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173975+0000", 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.852e-06 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173978+0000", 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5670000000000002e-06 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217784+0000", 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043805561 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217853+0000", 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9554999999999994e-05 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217913+0000", 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9435999999999998e-05 2024-09-16T14:08:00.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86514 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173980+0000", 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.481612079, 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044132973999999998, 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86514 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173980+0000", 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173980+0000", 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173980+0000", 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3200000000000001e-07 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173982+0000", 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173987+0000", 2024-09-16T14:08:00.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3159999999999996e-06 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173989+0000", 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-06 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217953+0000", 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043963277000000002 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218039+0000", 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6525000000000003e-05 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218113+0000", 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3929000000000003e-05 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.677 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86533 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 150528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173991+0000", 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4816007019999999, 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044265158999999998, 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86533 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173991+0000", 2024-09-16T14:08:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173991+0000", 2024-09-16T14:08:00.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173992+0000", 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4200000000000002e-07 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173993+0000", 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.331e-06 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173996+0000", 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9079999999999999e-06 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173998+0000", 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2869999999999999e-06 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218144+0000", 2024-09-16T14:08:00.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044145898000000003 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218211+0000", 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6531000000000004e-05 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218257+0000", 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86542 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 227328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174001+0000", 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4815910159999999, 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044461285000000003, 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.683 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86542 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174001+0000", 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174001+0000", 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174001+0000", 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4400000000000001e-07 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174003+0000", 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3069999999999999e-06 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174006+0000", 2024-09-16T14:08:00.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.943e-06 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174008+0000", 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.295e-06 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218297+0000", 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044289156000000003 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218391+0000", 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4296000000000007e-05 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218462+0000", 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1044000000000006e-05 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.687 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86559 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 143360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174021+0000", 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4815715270000001, 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044581127999999998, 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86559 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174021+0000", 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174021+0000", 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174021+0000", 2024-09-16T14:08:00.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2599999999999998e-07 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174028+0000", 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174038+0000", 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0077e-05 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174042+0000", 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.031e-06 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218496+0000", 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044453766999999998 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218541+0000", 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5055000000000001e-05 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218602+0000", 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1086000000000004e-05 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86565 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174046+0000", 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4815459280000001, 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044772340000000001, 2024-09-16T14:08:00.692 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86565 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174046+0000", 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174046+0000", 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174047+0000", 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7700000000000005e-07 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174050+0000", 2024-09-16T14:08:00.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7650000000000002e-06 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174059+0000", 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0070000000000003e-06 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174063+0000", 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4440000000000001e-06 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218643+0000", 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044579675999999999 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218740+0000", 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7770999999999996e-05 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218819+0000", 2024-09-16T14:08:00.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8200000000000003e-05 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86568 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174067+0000", 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4815250230000001, 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044897320999999997, 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86568 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174067+0000", 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174067+0000", 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174068+0000", 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174070+0000", 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2230000000000001e-06 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174078+0000", 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2779999999999993e-06 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174081+0000", 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8849999999999999e-06 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218841+0000", 2024-09-16T14:08:00.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044760507999999997 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218906+0000", 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4336000000000002e-05 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218964+0000", 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8717e-05 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.701 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86587 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 184320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174084+0000", 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4815086559999999, 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045073095000000001, 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.702 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86587 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174084+0000", 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174084+0000", 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174084+0000", 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8700000000000002e-07 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174085+0000", 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.372e-06 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174088+0000", 2024-09-16T14:08:00.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8559999999999998e-06 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174090+0000", 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2979999999999999e-06 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219002+0000", 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044911590000000001 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219081+0000", 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9095000000000001e-05 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219157+0000", 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5597000000000001e-05 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86593 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 261120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174092+0000", 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.481499865, 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045174586000000003, 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86593 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174092+0000", 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174092+0000", 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174093+0000", 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:08:00.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174094+0000", 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2470000000000001e-06 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174097+0000", 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9179999999999998e-06 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174099+0000", 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.323e-06 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219180+0000", 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045080536999999997 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219224+0000", 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4044e-05 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219267+0000", 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3260000000000003e-05 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86609 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 148480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174101+0000", 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4814911500000001, 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045401895999999997, 2024-09-16T14:08:00.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86609 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174101+0000", 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174101+0000", 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174101+0000", 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.41e-07 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174103+0000", 2024-09-16T14:08:00.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3820000000000001e-06 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174108+0000", 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3709999999999999e-06 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174111+0000", 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4700000000000001e-06 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219306+0000", 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045195573000000003 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219389+0000", 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2620000000000005e-05 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219503+0000", 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114239 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86615 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174113+0000", 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4814796960000001, 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045508728999999998, 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.716 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86615 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174113+0000", 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174113+0000", 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174113+0000", 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4400000000000001e-07 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174114+0000", 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0920000000000001e-06 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174119+0000", 2024-09-16T14:08:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174121+0000", 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.345e-06 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219512+0000", 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045390527999999999 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219567+0000", 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4892999999999998e-05 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219621+0000", 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4376999999999998e-05 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.720 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86640 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 189440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174124+0000", 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4814685160000001, 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045697645000000002, 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86640 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174124+0000", 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174124+0000", 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174124+0000", 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:08:00.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174125+0000", 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1570000000000001e-06 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174130+0000", 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0560000000000004e-06 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174133+0000", 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.841e-06 2024-09-16T14:08:00.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219661+0000", 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045528179000000002 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219751+0000", 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9903999999999998e-05 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219821+0000", 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0251000000000002e-05 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86671 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 196608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174135+0000", 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.481457061, 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045818207999999999, 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86671 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174135+0000", 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174135+0000", 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174135+0000", 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7799999999999997e-07 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174137+0000", 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.246e-06 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174140+0000", 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9179999999999998e-06 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174142+0000", 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5129999999999999e-06 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219854+0000", 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045711839999999997 2024-09-16T14:08:00.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219908+0000", 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4410999999999998e-05 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219953+0000", 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5002e-05 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86686 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 160768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176387+0000", 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.479205114, 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043740423, 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86686 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176387+0000", 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176387+0000", 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176388+0000", 2024-09-16T14:08:00.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4e-07 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176393+0000", 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7539999999999997e-06 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176410+0000", 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7345000000000001e-05 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176417+0000", 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.153e-06 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219987+0000", 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043570080999999997 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220062+0000", 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4617999999999996e-05 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220128+0000", 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5531999999999995e-05 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86693 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176424+0000", 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4791678420000001, 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043897472, 2024-09-16T14:08:00.734 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86693 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176424+0000", 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176424+0000", 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176425+0000", 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0500000000000004e-07 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176428+0000", 2024-09-16T14:08:00.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.678e-06 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176434+0000", 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4080000000000003e-06 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176439+0000", 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5700000000000003e-06 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220161+0000", 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043722703000000002 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220244+0000", 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3015000000000004e-05 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220322+0000", 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7593000000000001e-05 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86715 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176442+0000", 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4791497040000001, 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044069325999999999, 2024-09-16T14:08:00.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86715 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176442+0000", 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176442+0000", 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176443+0000", 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.03e-07 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176445+0000", 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.029e-06 2024-09-16T14:08:00.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176451+0000", 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176455+0000", 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220367+0000", 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043912400999999997 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220455+0000", 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8473000000000005e-05 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220512+0000", 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.643e-05 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.744 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86737 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176567+0000", 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.479024814, 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044084441000000002, 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86737 2024-09-16T14:08:00.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176567+0000", 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176567+0000", 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176568+0000", 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3500000000000005e-07 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176572+0000", 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7290000000000002e-06 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176585+0000", 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2961e-05 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176592+0000", 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7009999999999999e-06 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220545+0000", 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043953010000000001 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220602+0000", 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:08:00.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220652+0000", 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0142999999999997e-05 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86742 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176598+0000", 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.478994221, 2024-09-16T14:08:00.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0442216, 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86742 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176598+0000", 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176598+0000", 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176598+0000", 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7800000000000002e-07 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176601+0000", 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.604e-06 2024-09-16T14:08:00.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176608+0000", 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7329999999999997e-06 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176612+0000", 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2560000000000004e-06 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:00.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220671+0000", 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044059039000000001 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220747+0000", 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6196e-05 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220820+0000", 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2293999999999996e-05 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:00.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:00.754 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:00.755 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:00.755 INFO:teuthology.orchestra.run.smithi079.stdout:2514 write 0xc6f9728 thru 0xc705a5c (0xc335 bytes) 2024-09-16T14:08:00.755 INFO:teuthology.orchestra.run.smithi079.stdout:2516 write 0x75d7131 thru 0x75df4c2 (0x8392 bytes) 2024-09-16T14:08:00.756 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:00.756 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:00.757 INFO:teuthology.orchestra.run.smithi079.stdout:2518 write 0x5cb789c thru 0x5cb7f38 (0x69d bytes) 2024-09-16T14:08:00.757 INFO:teuthology.orchestra.run.smithi079.stdout:2519 read 0x74489b7 thru 0x744cb17 (0x4161 bytes) 2024-09-16T14:08:00.757 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:00.758 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:00.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:00.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:00.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:00.758 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:00.759 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:00.759 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:00.759 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:00.759 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:00.759 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:00.760 INFO:teuthology.orchestra.run.smithi079.stdout:2520 trunc from 0xc705a5d to 0x55567c6 2024-09-16T14:08:00.760 INFO:teuthology.orchestra.run.smithi079.stdout:2521 punch from 0x4cd62da to 0x4cd89d1, (0x26f7 bytes) 2024-09-16T14:08:00.760 INFO:teuthology.orchestra.run.smithi079.stdout:2522 trunc from 0x55567c6 to 0x6dbd908 2024-09-16T14:08:00.760 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:00.749+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:00.794 INFO:teuthology.orchestra.run.smithi079.stdout:2523 writesame 0xeadb954 thru 0xeaddaca data_size 0x1(0x2177 bytes) 2024-09-16T14:08:00.856 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:00.850+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:00.932 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:00.932 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:00.945 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:00.957 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:00.949+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:01.004 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:08:01.063 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:01.057+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:01.066 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.066 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.078 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:01.159 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:01.154+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:01.177 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.177 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.186 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:01.261 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:01.253+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:01.276 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.276 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.285 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:01.362 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:01.354+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:01.381 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.381 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.382 INFO:teuthology.orchestra.run.smithi079.stdout: marking collection for removal 2024-09-16T14:08:01.382 INFO:teuthology.orchestra.run.smithi079.stdout:setting '_remove' omap key 2024-09-16T14:08:01.389 INFO:teuthology.orchestra.run.smithi079.stdout:finish_remove_pgs 2.5_head removing 2.5 2024-09-16T14:08:01.391 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:01.392 INFO:teuthology.orchestra.run.smithi079.stdout:Remove successful 2024-09-16T14:08:01.463 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:01.458+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:01.544 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.544 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:01.563 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:01.558+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:01.655 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.655 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:01.665 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:01.658+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:01.758 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.759 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.766 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:01.762+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:01.767 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:01.863 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.863 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.867 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:01.862+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:01.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:01.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.5.3 2024-09-16T14:08:01.968 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:01.962+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:01.979 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:01.979 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:01.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:02.069 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:02.062+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:02.073 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:02.073 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:02.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:02.171 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:02.165+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:02.176 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:02.177 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:02.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:02.218 INFO:teuthology.orchestra.run.smithi079.stdout:2524 write 0x157a356 thru 0x157b02d (0xcd8 bytes) 2024-09-16T14:08:02.271 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:02.266+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:02.328 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:02.328 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:02.336 INFO:teuthology.orchestra.run.smithi079.stdout:2525 read 0x7594858 thru 0x7596b07 (0x22b0 bytes) 2024-09-16T14:08:02.336 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.337 INFO:teuthology.orchestra.run.smithi079.stdout:2529 read 0x5ddbb17 thru 0x5dddcff (0x21e9 bytes) 2024-09-16T14:08:02.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:02.338 INFO:teuthology.orchestra.run.smithi079.stdout:2530 write 0xbb5407b thru 0xbb5633e (0x22c4 bytes) 2024-09-16T14:08:02.344 INFO:teuthology.orchestra.run.smithi079.stdout:2531 writesame 0x12af37 thru 0x130656 data_size 0x52(0x5720 bytes) 2024-09-16T14:08:02.348 INFO:teuthology.orchestra.run.smithi079.stdout:2532 read 0xdc4a095 thru 0xdc50dac (0x6d18 bytes) 2024-09-16T14:08:02.348 INFO:teuthology.orchestra.run.smithi079.stdout:2534 write 0x2c15aab thru 0x2c1e03c (0x8592 bytes) 2024-09-16T14:08:02.372 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:02.366+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:02.437 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:02.437 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:02.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:02.473 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:02.466+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:02.512 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.512 INFO:teuthology.orchestra.run.smithi079.stdout:2537 write 0xa9e5f89 thru 0xa9edabb (0x7b33 bytes) 2024-09-16T14:08:02.543 INFO:teuthology.orchestra.run.smithi079.stdout:2538 trunc from 0xeaddacb to 0x26c6a92 2024-09-16T14:08:02.550 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:02.550 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:02.562 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:02.569 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.569 INFO:teuthology.orchestra.run.smithi079.stdout:2540 read 0x1574c82 thru 0x157b636 (0x69b5 bytes) 2024-09-16T14:08:02.573 INFO:teuthology.orchestra.run.smithi079.stdout:2544 trunc from 0x26c6a92 to 0x4db0954 2024-09-16T14:08:02.582 INFO:teuthology.orchestra.run.smithi079.stdout:2545 punch from 0x1062351 to 0x106a62d, (0x82dc bytes) 2024-09-16T14:08:02.582 INFO:teuthology.orchestra.run.smithi079.stdout:2548 writesame 0x2403712 thru 0x2408a6d data_size 0x16(0x535c bytes) 2024-09-16T14:08:02.598 INFO:teuthology.orchestra.run.smithi079.stdout:2550 writesame 0x6203a43 thru 0x620f022 data_size 0x50(0xb5e0 bytes) 2024-09-16T14:08:02.598 INFO:teuthology.orchestra.run.smithi079.stdout:2551 punch from 0x42797fa to 0x428732b, (0xdb31 bytes) 2024-09-16T14:08:02.598 INFO:teuthology.orchestra.run.smithi079.stdout:2554 read 0x31722ee thru 0x3176e4d (0x4b60 bytes) 2024-09-16T14:08:02.598 INFO:teuthology.orchestra.run.smithi079.stdout:2555 punch from 0x1b7313a to 0x1b7965f, (0x6525 bytes) 2024-09-16T14:08:02.598 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.598 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.607 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:02.602+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:02.608 INFO:teuthology.orchestra.run.smithi079.stdout:2558 trunc from 0x620f023 to 0x592062 2024-09-16T14:08:02.635 INFO:teuthology.orchestra.run.smithi079.stdout:2559 punch from 0x2f4b91 to 0x2f9dcc, (0x523b bytes) 2024-09-16T14:08:02.635 INFO:teuthology.orchestra.run.smithi079.stdout:2560 punch from 0x79811 to 0x7d668, (0x3e57 bytes) 2024-09-16T14:08:02.635 INFO:teuthology.orchestra.run.smithi079.stdout:2561 read 0x4c021d thru 0x4c7ff9 (0x7ddd bytes) 2024-09-16T14:08:02.636 INFO:teuthology.orchestra.run.smithi079.stdout:2564 punch from 0x34944c to 0x35551c, (0xc0d0 bytes) 2024-09-16T14:08:02.637 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.637 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.656 INFO:teuthology.orchestra.run.smithi079.stdout:2569 write 0x4a73e4c thru 0x4a7a415 (0x65ca bytes) 2024-09-16T14:08:02.662 INFO:teuthology.orchestra.run.smithi079.stdout:2571 punch from 0x1c4fda0 to 0x1c5d11c, (0xd37c bytes) 2024-09-16T14:08:02.662 INFO:teuthology.orchestra.run.smithi079.stdout:2572 read 0x897efe thru 0x899c2a (0x1d2d bytes) 2024-09-16T14:08:02.662 INFO:teuthology.orchestra.run.smithi079.stdout:2573 read 0x2efed6e thru 0x2f0d110 (0xe3a3 bytes) 2024-09-16T14:08:02.666 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:02.666 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:02.670 INFO:teuthology.orchestra.run.smithi079.stdout:2575 trunc from 0x4a7a416 to 0x8593eff 2024-09-16T14:08:02.675 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:02.666+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:02.675 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:02.681 INFO:teuthology.orchestra.run.smithi079.stdout:2577 write 0x33add7a thru 0x33ba224 (0xc4ab bytes) 2024-09-16T14:08:02.688 INFO:teuthology.orchestra.run.smithi079.stdout:2578 punch from 0x6f2d7bc to 0x6f33120, (0x5964 bytes) 2024-09-16T14:08:02.693 INFO:teuthology.orchestra.run.smithi079.stdout:2579 trunc from 0x8593eff to 0x5cf2c38 2024-09-16T14:08:02.705 INFO:teuthology.orchestra.run.smithi079.stdout:2580 write 0x38c4eb1 thru 0x38cf605 (0xa755 bytes) 2024-09-16T14:08:02.708 INFO:teuthology.orchestra.run.smithi079.stdout:2581 trunc from 0x5cf2c38 to 0x5dbbf0b 2024-09-16T14:08:02.721 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.721 INFO:teuthology.orchestra.run.smithi079.stdout:2584 writesame 0x89cb7 thru 0x96f04 data_size 0x1b(0xd24e bytes) 2024-09-16T14:08:02.725 INFO:teuthology.orchestra.run.smithi079.stdout:2585 writesame 0x29da14f thru 0x29e461a data_size 0x6a(0xa4cc bytes) 2024-09-16T14:08:02.755 INFO:teuthology.orchestra.run.smithi079.stdout:2587 write 0xbdf1101 thru 0xbdf865f (0x755f bytes) 2024-09-16T14:08:02.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:02.762 INFO:teuthology.orchestra.run.smithi079.stdout:2588 write 0x5d9598e thru 0x5d9e126 (0x8799 bytes) 2024-09-16T14:08:02.768 INFO:teuthology.orchestra.run.smithi079.stdout:2589 write 0x27e9d8 thru 0x2843e3 (0x5a0c bytes) 2024-09-16T14:08:02.770 INFO:teuthology.orchestra.run.smithi079.stdout:2591 read 0x6850fa7 thru 0x685a78d (0x97e7 bytes) 2024-09-16T14:08:02.770 INFO:teuthology.orchestra.run.smithi079.stdout:2593 writesame 0xa849a thru 0xaa5a4 data_size 0x1(0x210b bytes) 2024-09-16T14:08:02.774 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:02.769+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:02.779 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:02.780 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:02.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:02.793 INFO:teuthology.orchestra.run.smithi079.stdout:2594 trunc from 0xbdf8660 to 0x462e028 2024-09-16T14:08:02.805 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.805 INFO:teuthology.orchestra.run.smithi079.stdout:2596 trunc from 0x462e028 to 0x4225acd 2024-09-16T14:08:02.815 INFO:teuthology.orchestra.run.smithi079.stdout:2597 read 0xc4c1d8 thru 0xc5810f (0xbf38 bytes) 2024-09-16T14:08:02.816 INFO:teuthology.orchestra.run.smithi079.stdout:2599 read 0x337d243 thru 0x338c8f4 (0xf6b2 bytes) 2024-09-16T14:08:02.817 INFO:teuthology.orchestra.run.smithi079.stdout:2600 punch from 0x2b0d4af to 0x2b150c8, (0x7c19 bytes) 2024-09-16T14:08:02.817 INFO:teuthology.orchestra.run.smithi079.stdout:2601 read 0x309174c thru 0x30a0377 (0xec2c bytes) 2024-09-16T14:08:02.835 INFO:teuthology.orchestra.run.smithi079.stdout:2603 writesame 0x829cc83 thru 0x82a5e11 data_size 0x1(0x918f bytes) 2024-09-16T14:08:02.847 INFO:teuthology.orchestra.run.smithi079.stdout:2604 write 0xac8c932 thru 0xac9c278 (0xf947 bytes) 2024-09-16T14:08:02.853 INFO:teuthology.orchestra.run.smithi079.stdout:2605 write 0x30c3faf thru 0x30c7135 (0x3187 bytes) 2024-09-16T14:08:02.856 INFO:teuthology.orchestra.run.smithi079.stdout:2606 read 0x3a15a23 thru 0x3a1743c (0x1a1a bytes) 2024-09-16T14:08:02.864 INFO:teuthology.orchestra.run.smithi079.stdout:2608 trunc from 0xac9c279 to 0x6c3fbc2 2024-09-16T14:08:02.876 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:02.870+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:02.881 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.882 INFO:teuthology.orchestra.run.smithi079.stdout:2611 read 0x50bba89 thru 0x50be2d4 (0x284c bytes) 2024-09-16T14:08:02.882 INFO:teuthology.orchestra.run.smithi079.stdout:2613 write 0x4fbbed1 thru 0x4fca906 (0xea36 bytes) 2024-09-16T14:08:02.884 INFO:teuthology.orchestra.run.smithi079.stdout:2615 read 0x194c2a5 thru 0x19513a9 (0x5105 bytes) 2024-09-16T14:08:02.888 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:02.888 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:02.896 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:02.897 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:02.898 INFO:teuthology.orchestra.run.smithi079.stdout:2616 writesame 0xaf3c8ea thru 0xaf4443c data_size 0x1(0x7b53 bytes) 2024-09-16T14:08:02.898 INFO:teuthology.orchestra.run.smithi079.stdout:2617 write 0x84f39a0 thru 0x84f87ac (0x4e0d bytes) 2024-09-16T14:08:02.905 INFO:teuthology.orchestra.run.smithi079.stdout:2618 read 0x5c4d406 thru 0x5c4e431 (0x102c bytes) 2024-09-16T14:08:02.905 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.913 INFO:teuthology.orchestra.run.smithi079.stdout:2620 trunc from 0xaf4443d to 0x57a2bd4 2024-09-16T14:08:02.921 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:02.934 INFO:teuthology.orchestra.run.smithi079.stdout:2621 writesame 0x1598dc6 thru 0x15a5ff2 data_size 0x55(0xd22d bytes) 2024-09-16T14:08:02.941 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.944 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:02.953 INFO:teuthology.orchestra.run.smithi079.stdout:2623 write 0x6c900ed thru 0x6c9b747 (0xb65b bytes) 2024-09-16T14:08:02.959 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:02.959 INFO:teuthology.orchestra.run.smithi079.stdout:2626 punch from 0x2b81e07 to 0x2b8f955, (0xdb4e bytes) 2024-09-16T14:08:02.959 INFO:teuthology.orchestra.run.smithi079.stdout:2627 clone 22 order 21 su 524288 sc 2 2024-09-16T14:08:02.966 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:02.977 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:02.970+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:02.988 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:02.996 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:02.996 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:02.996 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:02.996 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:03.005 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:03.013 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:03.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:08:03.037 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:03.037 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:03.040 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:03.040 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:03.079 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:03.074+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:03.135 INFO:teuthology.orchestra.run.smithi079.stdout:get_pg_num_history pg_num_history pg_num_history(e58 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-16T14:08:03.135 INFO:teuthology.orchestra.run.smithi079.stdout:Importing pgid 2.5 2024-09-16T14:08:03.135 INFO:teuthology.orchestra.run.smithi079.stdout:write_pg epoch 25 info 2.5( empty local-lis/les=19/20 n=0 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-16T14:08:03.135 INFO:teuthology.orchestra.run.smithi079.stdout:Import successful 2024-09-16T14:08:03.155 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:03.155 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:03.164 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:03.179 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:03.174+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:03.230 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone21 from 0x6c9b748 (overlap 0x592062) to 0x8e9a3 2024-09-16T14:08:03.268 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:03.268 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:03.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:03.268 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:03.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.268 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86470 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 258048~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.268 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173921+0000", 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0902881520000003, 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043647279999999997, 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86470 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173921+0000", 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173921+0000", 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173922+0000", 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173924+0000", 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.086e-06 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173933+0000", 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3519999999999996e-06 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173938+0000", 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5310000000000003e-06 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217431+0000", 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043493433999999997 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217520+0000", 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9172999999999995e-05 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217569+0000", 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8276000000000003e-05 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86482 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 145408~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173943+0000", 2024-09-16T14:08:03.274 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0902666920000001, 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043821633999999998, 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86482 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173943+0000", 2024-09-16T14:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173943+0000", 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173943+0000", 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173946+0000", 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2220000000000001e-06 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173955+0000", 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4970000000000007e-06 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173963+0000", 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1540000000000002e-06 2024-09-16T14:08:03.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217604+0000", 2024-09-16T14:08:03.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043641081999999998 2024-09-16T14:08:03.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217692+0000", 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8109000000000006e-05 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217765+0000", 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2127999999999998e-05 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86493 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 222208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173967+0000", 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0902421210000002, 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043945154, 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86493 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173967+0000", 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173967+0000", 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173968+0000", 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5199999999999998e-07 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173970+0000", 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8309999999999999e-06 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173975+0000", 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.852e-06 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173978+0000", 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5670000000000002e-06 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217784+0000", 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043805561 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217853+0000", 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9554999999999994e-05 2024-09-16T14:08:03.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217913+0000", 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9435999999999998e-05 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86514 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173980+0000", 2024-09-16T14:08:03.285 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0902294430000001, 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044132973999999998, 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86514 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173980+0000", 2024-09-16T14:08:03.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173980+0000", 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173980+0000", 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3200000000000001e-07 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173982+0000", 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173987+0000", 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3159999999999996e-06 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173989+0000", 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4049999999999998e-06 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.217953+0000", 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043963277000000002 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218039+0000", 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6525000000000003e-05 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218113+0000", 2024-09-16T14:08:03.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3929000000000003e-05 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86533 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 150528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.173991+0000", 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0902180660000003, 2024-09-16T14:08:03.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044265158999999998, 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86533 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173991+0000", 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173991+0000", 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173992+0000", 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4200000000000002e-07 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173993+0000", 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.331e-06 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173996+0000", 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9079999999999999e-06 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.173998+0000", 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2869999999999999e-06 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218144+0000", 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044145898000000003 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218211+0000", 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6531000000000004e-05 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218257+0000", 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T14:08:03.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86542 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 227328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174001+0000", 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.09020838, 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044461285000000003, 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.295 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86542 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174001+0000", 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174001+0000", 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174001+0000", 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4400000000000001e-07 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174003+0000", 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3069999999999999e-06 2024-09-16T14:08:03.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174006+0000", 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.943e-06 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174008+0000", 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.295e-06 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218297+0000", 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044289156000000003 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218391+0000", 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4296000000000007e-05 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218462+0000", 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1044000000000006e-05 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86559 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 143360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174021+0000", 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0901888910000004, 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044581127999999998, 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86559 2024-09-16T14:08:03.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174021+0000", 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174021+0000", 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174021+0000", 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2599999999999998e-07 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174028+0000", 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174038+0000", 2024-09-16T14:08:03.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0077e-05 2024-09-16T14:08:03.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174042+0000", 2024-09-16T14:08:03.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.031e-06 2024-09-16T14:08:03.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218496+0000", 2024-09-16T14:08:03.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044453766999999998 2024-09-16T14:08:03.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218541+0000", 2024-09-16T14:08:03.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5055000000000001e-05 2024-09-16T14:08:03.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218602+0000", 2024-09-16T14:08:03.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1086000000000004e-05 2024-09-16T14:08:03.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86565 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 220160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174046+0000", 2024-09-16T14:08:03.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0901632919999997, 2024-09-16T14:08:03.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044772340000000001, 2024-09-16T14:08:03.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86565 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174046+0000", 2024-09-16T14:08:03.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174046+0000", 2024-09-16T14:08:03.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174047+0000", 2024-09-16T14:08:03.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7700000000000005e-07 2024-09-16T14:08:03.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174050+0000", 2024-09-16T14:08:03.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7650000000000002e-06 2024-09-16T14:08:03.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174059+0000", 2024-09-16T14:08:03.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0070000000000003e-06 2024-09-16T14:08:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174063+0000", 2024-09-16T14:08:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4440000000000001e-06 2024-09-16T14:08:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218643+0000", 2024-09-16T14:08:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044579675999999999 2024-09-16T14:08:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218740+0000", 2024-09-16T14:08:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7770999999999996e-05 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218819+0000", 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8200000000000003e-05 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86568 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174067+0000", 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0901423870000002, 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044897320999999997, 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86568 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174067+0000", 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174067+0000", 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174068+0000", 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174070+0000", 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2230000000000001e-06 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174078+0000", 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2779999999999993e-06 2024-09-16T14:08:03.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174081+0000", 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8849999999999999e-06 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218841+0000", 2024-09-16T14:08:03.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044760507999999997 2024-09-16T14:08:03.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218906+0000", 2024-09-16T14:08:03.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4336000000000002e-05 2024-09-16T14:08:03.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.218964+0000", 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8717e-05 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86587 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 184320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174084+0000", 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0901260199999996, 2024-09-16T14:08:03.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045073095000000001, 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86587 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174084+0000", 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174084+0000", 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174084+0000", 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8700000000000002e-07 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174085+0000", 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.372e-06 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174088+0000", 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8559999999999998e-06 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174090+0000", 2024-09-16T14:08:03.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2979999999999999e-06 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219002+0000", 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044911590000000001 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219081+0000", 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9095000000000001e-05 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219157+0000", 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5597000000000001e-05 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86593 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 261120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174092+0000", 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0901172289999996, 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045174586000000003, 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.325 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86593 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174092+0000", 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174092+0000", 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174093+0000", 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174094+0000", 2024-09-16T14:08:03.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2470000000000001e-06 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174097+0000", 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9179999999999998e-06 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174099+0000", 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.323e-06 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219180+0000", 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045080536999999997 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219224+0000", 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4044e-05 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219267+0000", 2024-09-16T14:08:03.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3260000000000003e-05 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86609 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 148480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174101+0000", 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0901085139999998, 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045401895999999997, 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86609 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174101+0000", 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174101+0000", 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174101+0000", 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.41e-07 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174103+0000", 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3820000000000001e-06 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174108+0000", 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3709999999999999e-06 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174111+0000", 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4700000000000001e-06 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219306+0000", 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045195573000000003 2024-09-16T14:08:03.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219389+0000", 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2620000000000005e-05 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219503+0000", 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114239 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86615 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174113+0000", 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0900970599999997, 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045508728999999998, 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86615 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174113+0000", 2024-09-16T14:08:03.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174113+0000", 2024-09-16T14:08:03.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174113+0000", 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4400000000000001e-07 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174114+0000", 2024-09-16T14:08:03.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0920000000000001e-06 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174119+0000", 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174121+0000", 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.345e-06 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219512+0000", 2024-09-16T14:08:03.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045390527999999999 2024-09-16T14:08:03.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219567+0000", 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4892999999999998e-05 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219621+0000", 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4376999999999998e-05 2024-09-16T14:08:03.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.341 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86640 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 189440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.341 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174124+0000", 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0900858800000002, 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045697645000000002, 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86640 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.342 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174124+0000", 2024-09-16T14:08:03.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174124+0000", 2024-09-16T14:08:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174124+0000", 2024-09-16T14:08:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:08:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174125+0000", 2024-09-16T14:08:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1570000000000001e-06 2024-09-16T14:08:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174130+0000", 2024-09-16T14:08:03.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0560000000000004e-06 2024-09-16T14:08:03.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174133+0000", 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.841e-06 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219661+0000", 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045528179000000002 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219751+0000", 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9903999999999998e-05 2024-09-16T14:08:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219821+0000", 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0251000000000002e-05 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86671 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 196608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.174135+0000", 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0900744250000001, 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045818207999999999, 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86671 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174135+0000", 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174135+0000", 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174135+0000", 2024-09-16T14:08:03.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7799999999999997e-07 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174137+0000", 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.246e-06 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174140+0000", 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9179999999999998e-06 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.174142+0000", 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5129999999999999e-06 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219854+0000", 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045711839999999997 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219908+0000", 2024-09-16T14:08:03.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4410999999999998e-05 2024-09-16T14:08:03.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219953+0000", 2024-09-16T14:08:03.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5002e-05 2024-09-16T14:08:03.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.354 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.354 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86686 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 160768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.354 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176387+0000", 2024-09-16T14:08:03.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0878224779999996, 2024-09-16T14:08:03.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043740423, 2024-09-16T14:08:03.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.355 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86686 2024-09-16T14:08:03.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176387+0000", 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176387+0000", 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176388+0000", 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4e-07 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176393+0000", 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7539999999999997e-06 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176410+0000", 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7345000000000001e-05 2024-09-16T14:08:03.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176417+0000", 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.153e-06 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.219987+0000", 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043570080999999997 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220062+0000", 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4617999999999996e-05 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220128+0000", 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5531999999999995e-05 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86693 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.360 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176424+0000", 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0877852060000004, 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043897472, 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86693 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176424+0000", 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176424+0000", 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176425+0000", 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0500000000000004e-07 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176428+0000", 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.678e-06 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176434+0000", 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4080000000000003e-06 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176439+0000", 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5700000000000003e-06 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220161+0000", 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043722703000000002 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220244+0000", 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3015000000000004e-05 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220322+0000", 2024-09-16T14:08:03.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7593000000000001e-05 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86715 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 201728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176442+0000", 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0877670679999998, 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044069325999999999, 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86715 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176442+0000", 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176442+0000", 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176443+0000", 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.03e-07 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176445+0000", 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.029e-06 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176451+0000", 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176455+0000", 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220367+0000", 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043912400999999997 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220455+0000", 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8473000000000005e-05 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220512+0000", 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.643e-05 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.369 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86737 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 165888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176567+0000", 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0876421780000003, 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044084441000000002, 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86737 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176567+0000", 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176567+0000", 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176568+0000", 2024-09-16T14:08:03.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3500000000000005e-07 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176572+0000", 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7290000000000002e-06 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176585+0000", 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2961e-05 2024-09-16T14:08:03.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176592+0000", 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7009999999999999e-06 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220545+0000", 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043953010000000001 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220602+0000", 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7061999999999997e-05 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220652+0000", 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0142999999999997e-05 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:86742 3.1f 3:ff6c0c9f:::rbd_data.113619b3ee73.000000000000002f:16 [read 242688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.176598+0000", 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0876115850000003, 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0442216, 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 86742 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176598+0000", 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176598+0000", 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176598+0000", 2024-09-16T14:08:03.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7800000000000002e-07 2024-09-16T14:08:03.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176601+0000", 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.604e-06 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176608+0000", 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7329999999999997e-06 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.176612+0000", 2024-09-16T14:08:03.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2560000000000004e-06 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220671+0000", 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044059039000000001 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220747+0000", 2024-09-16T14:08:03.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6196e-05 2024-09-16T14:08:03.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.220820+0000", 2024-09-16T14:08:03.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2293999999999996e-05 2024-09-16T14:08:03.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.380 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.381 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.381 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:03.383 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:03.384 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:03.374+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:03.483 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:03.477+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:03.539 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:03.539 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:03.539 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:03.539 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:03.549 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:03.585 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:03.577+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:03.627 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-16T14:08:03.678 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:03.679 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:03.686 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:03.681+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:03.692 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:03.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /var/log/ceph/exp.2.5.3 2024-09-16T14:08:03.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:08:03.787 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:03.781+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:03.813 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:9779 3.7 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.516710029000002, 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.042761995999999997, 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 9779 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293648+0000", 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.293651+0000", 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.021e-06 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305709+0000", 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012058627000000001 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305754+0000", 2024-09-16T14:08:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4137000000000001e-05 2024-09-16T14:08:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305769+0000", 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5257e-05 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.305971+0000", 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000201833 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.306248+0000", 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027726999999999998 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.307018+0000", 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076971799999999996 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.329087+0000", 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022069787 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336319+0000", 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072311789999999999 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336375+0000", 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6406999999999997e-05 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336410+0000", 2024-09-16T14:08:03.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4759999999999999e-05 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12172 3.12 3:4dae129f:::rbd_data.11364aaed44f.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.263414740999998, 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.031007818999999999, 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12172 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546943+0000", 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546946+0000", 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2270000000000002e-06 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546958+0000", 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.173e-05 2024-09-16T14:08:03.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.546989+0000", 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1449000000000003e-05 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547001+0000", 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1216999999999999e-05 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547125+0000", 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012428299999999999 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.547345+0000", 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022038799999999999 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.555417+0000", 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080718119999999994 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577743+0000", 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02232599 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577765+0000", 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1675000000000002e-05 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577879+0000", 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011386400000000001 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.577951+0000", 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2183999999999999e-05 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14170 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.189481600000001, 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023454853000000001, 2024-09-16T14:08:03.827 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14170 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620876+0000", 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620877+0000", 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1500000000000005e-07 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620879+0000", 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0269999999999998e-06 2024-09-16T14:08:03.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620884+0000", 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7180000000000001e-06 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620891+0000", 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2319999999999998e-06 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644192+0000", 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023301776 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644255+0000", 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2948999999999996e-05 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644331+0000", 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5636000000000006e-05 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14172 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 344064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.189463571000001, 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023574827999999999, 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14172 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620894+0000", 2024-09-16T14:08:03.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620895+0000", 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4300000000000002e-07 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620897+0000", 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.939e-06 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620903+0000", 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7450000000000001e-06 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620908+0000", 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7769999999999999e-06 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644351+0000", 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023442411 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644388+0000", 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7441999999999998e-05 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644469+0000", 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0771000000000003e-05 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14175 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.189442988, 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023718466000000001, 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14175 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620915+0000", 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620927+0000", 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2113000000000001e-05 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620943+0000", 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620947+0000", 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5669999999999996e-06 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644494+0000", 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023546886999999999 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644563+0000", 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8485000000000006e-05 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644633+0000", 2024-09-16T14:08:03.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0529000000000002e-05 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14176 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 917504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.189405841999999, 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023877883999999999, 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14176 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620952+0000", 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9399999999999995e-07 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620954+0000", 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0200000000000001e-06 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620960+0000", 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8440000000000003e-06 2024-09-16T14:08:03.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620964+0000", 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7459999999999998e-06 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644666+0000", 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023702375000000001 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644720+0000", 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3813000000000001e-05 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644830+0000", 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109592 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14179 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:08:03.845 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.189386336999998, 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024008551999999999, 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14179 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620971+0000", 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620972+0000", 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1200000000000003e-07 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620981+0000", 2024-09-16T14:08:03.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4420000000000004e-06 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620988+0000", 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.297e-06 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620992+0000", 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8979999999999998e-06 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644865+0000", 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023872899999999999 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644910+0000", 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5794999999999998e-05 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.644980+0000", 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9708000000000004e-05 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.849 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:14180 3.7 3:e27ab6a2:::rbd_data.11364aaed44f.0000000000000000:c [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.189362194000001, 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024190943999999999, 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14180 2024-09-16T14:08:03.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620996+0000", 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0100000000000005e-07 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.620998+0000", 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9829999999999999e-06 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.621004+0000", 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8270000000000003e-06 2024-09-16T14:08:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.621009+0000", 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4530000000000001e-06 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645013+0000", 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024003295000000001 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645077+0000", 2024-09-16T14:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4139000000000004e-05 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:35.645187+0000", 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109746 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101613 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656214+0000", 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.154143799, 2024-09-16T14:08:03.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023667500000000001, 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101613 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656214+0000", 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656214+0000", 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656214+0000", 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4999999999999998e-07 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656216+0000", 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.99e-06 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656222+0000", 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9070000000000004e-06 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656226+0000", 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.332e-06 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679689+0000", 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023463639000000001 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679765+0000", 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6097e-05 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679881+0000", 2024-09-16T14:08:03.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116185 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101614 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656230+0000", 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15412817600000001, 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023866753000000001, 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.859 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101614 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656230+0000", 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656230+0000", 2024-09-16T14:08:03.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656230+0000", 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2599999999999998e-07 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656232+0000", 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.745e-06 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656237+0000", 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7069999999999997e-06 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656241+0000", 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5049999999999999e-06 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679906+0000", 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023665345000000001 2024-09-16T14:08:03.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679980+0000", 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3627999999999994e-05 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680096+0000", 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011649700000000001 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101615 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 55296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656245+0000", 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.154113204, 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024076732999999999, 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101615 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.864 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656245+0000", 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656245+0000", 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656245+0000", 2024-09-16T14:08:03.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656247+0000", 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.778e-06 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656252+0000", 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.592e-06 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656260+0000", 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999998e-06 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680125+0000", 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023865152000000001 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680203+0000", 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8411999999999994e-05 2024-09-16T14:08:03.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680321+0000", 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118108 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101616 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656263+0000", 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15409437100000001, 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024279508000000002, 2024-09-16T14:08:03.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101616 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656263+0000", 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656263+0000", 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656264+0000", 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656266+0000", 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.798e-06 2024-09-16T14:08:03.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656271+0000", 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6370000000000004e-06 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656275+0000", 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6059999999999999e-06 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680347+0000", 2024-09-16T14:08:03.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024072387000000001 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680424+0000", 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7031000000000002e-05 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680543+0000", 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118709 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101617 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656279+0000", 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15407911499999999, 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024484512999999999, 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101617 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656279+0000", 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656279+0000", 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656279+0000", 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:03.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656281+0000", 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.765e-06 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656288+0000", 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.559e-06 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656292+0000", 2024-09-16T14:08:03.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3189999999999999e-06 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680570+0000", 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024278399999999999 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680645+0000", 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4476999999999999e-05 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680763+0000", 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101618 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656296+0000", 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15406207699999999, 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024687367000000002, 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101618 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656296+0000", 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656296+0000", 2024-09-16T14:08:03.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656296+0000", 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7e-07 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656298+0000", 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9120000000000001e-06 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656308+0000", 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0261000000000001e-05 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656312+0000", 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6320000000000001e-06 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680790+0000", 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024477878000000002 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680864+0000", 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4317e-05 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680983+0000", 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101619 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656316+0000", 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.154041657, 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024888229000000001, 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101619 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656316+0000", 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656316+0000", 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656317+0000", 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656318+0000", 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.95e-06 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656324+0000", 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7819999999999999e-06 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656328+0000", 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0600000000000001e-06 2024-09-16T14:08:03.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681011+0000", 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024682353000000001 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681086+0000", 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5405000000000001e-05 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681204+0000", 2024-09-16T14:08:03.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118298 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101620 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656332+0000", 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15402570900000001, 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025089986000000002, 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.886 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101620 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656332+0000", 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656332+0000", 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656332+0000", 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6399999999999998e-07 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656335+0000", 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1430000000000001e-06 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656345+0000", 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0190000000000001e-05 2024-09-16T14:08:03.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656348+0000", 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6160000000000002e-06 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681231+0000", 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024883057 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681307+0000", 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6017000000000001e-05 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681422+0000", 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011459900000000001 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101621 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656352+0000", 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15400540300000001, 2024-09-16T14:08:03.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025285668000000001, 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101621 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656352+0000", 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656352+0000", 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656353+0000", 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1600000000000002e-07 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656355+0000", 2024-09-16T14:08:03.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.902e-06 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656363+0000", 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4880000000000007e-06 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656367+0000", 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5659999999999998e-06 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681450+0000", 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025083511999999999 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681526+0000", 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5946999999999996e-05 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681638+0000", 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111937 2024-09-16T14:08:03.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101622 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656371+0000", 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15398690600000001, 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025496218000000001, 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101622 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656371+0000", 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656371+0000", 2024-09-16T14:08:03.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656371+0000", 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5900000000000003e-07 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656378+0000", 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7349999999999999e-06 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656387+0000", 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8990000000000007e-06 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656391+0000", 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0720000000000001e-06 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681664+0000", 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025272664 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681737+0000", 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3491000000000005e-05 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681867+0000", 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012999800000000001 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101623 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656396+0000", 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.15396205700000001, 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025681583000000001, 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101623 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656396+0000", 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656396+0000", 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656396+0000", 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999998e-07 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656398+0000", 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1220000000000002e-06 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656407+0000", 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5949999999999999e-06 2024-09-16T14:08:03.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656410+0000", 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.585e-06 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681892+0000", 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025481588999999999 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681964+0000", 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1532000000000006e-05 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.682077+0000", 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011374099999999999 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101624 3.12 3:49fd99ed:::rbd_data.113618577e70.0000000000000009:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.656414+0000", 2024-09-16T14:08:03.904 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.153943303, 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025880796000000001, 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101624 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656414+0000", 2024-09-16T14:08:03.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656414+0000", 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656415+0000", 2024-09-16T14:08:03.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.84e-07 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656417+0000", 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.962e-06 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656423+0000", 2024-09-16T14:08:03.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8730000000000004e-06 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.656426+0000", 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4929999999999998e-06 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.682101+0000", 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025674694000000001 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:03.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.682177+0000", 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6007000000000006e-05 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.682295+0000", 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011838299999999999 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:03.909 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:03.911 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:03.911 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:03.905+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:04.011 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:04.006+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:04.030 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:08:04.030 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:04.030 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:04.030 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:04.030 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:04.047 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:04.113 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:04.105+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:04.148 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:04.148 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:04.148 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:04.148 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:04.148 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:04.159 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:04.166 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:08:04.166 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":66,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:08:04.000196+0000","last_up_change":"2024-09-16T14:07:53.884253+0000","last_in_change":"2024-09-16T14:06:42.748450+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:06:58.526935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T14:07:08.375919+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":22196},{"type":"v1","addr":"172.21.15.79:6817","nonce":22196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":22196},{"type":"v1","addr":"172.21.15.79:6819","nonce":22196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":22196},{"type":"v1","addr":"172.21.15.79:6824","nonce":22196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":22196},{"type":"v1","addr":"172.21.15.79:6821","nonce":22196}]},"public_addr":"172.21.15.79:6817/22196","cluster_addr":"172.21.15.79:6819/22196","heartbeat_back_addr":"172.21.15.79:6824/22196","heartbeat_front_addr":"172.21.15.79:6821/22196","state":["exists","up"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":22193},{"type":"v1","addr":"172.21.15.79:6801","nonce":22193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":22193},{"type":"v1","addr":"172.21.15.79:6803","nonce":22193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":22193},{"type":"v1","addr":"172.21.15.79:6807","nonce":22193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":22193},{"type":"v1","addr":"172.21.15.79:6805","nonce":22193}]},"public_addr":"172.21.15.79:6801/22193","cluster_addr":"172.21.15.79:6803/22193","heartbeat_back_addr":"172.21.15.79:6807/22193","heartbeat_front_addr":"172.21.15.79:6805/22193","state":["exists"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":0.242828369140625,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":59},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:08:04.187 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T14:06:52.822826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T14:06:58.526935+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T14:07:08.375919+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 65, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-16T14:08:04.187 DEBUG:teuthology.orchestra.run.smithi079:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op apply-layout-settings --pool .mgr 2024-09-16T14:08:04.215 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:04.205+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:04.271 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:04.271 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:04.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:04.271 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:04.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.271 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.857256077999999, 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:08:04.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:08:04.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:08:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:04.279 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.019293755, 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:08:04.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:08:04.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89383 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.248732+0000", 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0184604889999997, 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011214494, 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89383 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248732+0000", 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248732+0000", 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248732+0000", 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1799999999999995e-07 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248734+0000", 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6589999999999999e-06 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248743+0000", 2024-09-16T14:08:04.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9900000000000003e-06 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248746+0000", 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.292e-06 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.259715+0000", 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010968894 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.259823+0000", 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108039 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.259946+0000", 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012310199999999999 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89384 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 184320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.248749+0000", 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0184434720000004, 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011463403, 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89384 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248749+0000", 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248749+0000", 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248749+0000", 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8700000000000002e-07 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248751+0000", 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.497e-06 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248754+0000", 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2629999999999999e-06 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248756+0000", 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:04.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.259990+0000", 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011233304 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260101+0000", 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110998 2024-09-16T14:08:04.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260212+0000", 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011179999999999999 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89385 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 208896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.248759+0000", 2024-09-16T14:08:04.295 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0184338630000003, 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011638331999999999, 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89385 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248759+0000", 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248759+0000", 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248759+0000", 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1300000000000001e-07 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248762+0000", 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5579999999999999e-06 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248765+0000", 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0709999999999999e-06 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248768+0000", 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9960000000000002e-06 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260250+0000", 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011482711 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260315+0000", 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4443000000000006e-05 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260397+0000", 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-05 2024-09-16T14:08:04.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89386 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.248770+0000", 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0184228470000001, 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011783052, 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89386 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248770+0000", 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248770+0000", 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248770+0000", 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248773+0000", 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0240000000000002e-06 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248776+0000", 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1370000000000002e-06 2024-09-16T14:08:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248778+0000", 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2290000000000002e-06 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260431+0000", 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011653030999999999 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260490+0000", 2024-09-16T14:08:04.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8309e-05 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260553+0000", 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3075000000000002e-05 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89387 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.248780+0000", 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0184121309999998, 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011916863999999999, 2024-09-16T14:08:04.304 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89387 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248780+0000", 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248780+0000", 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248781+0000", 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5600000000000002e-07 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248782+0000", 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.251e-06 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248786+0000", 2024-09-16T14:08:04.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1030000000000004e-06 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.248788+0000", 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260585+0000", 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011797337999999999 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260649+0000", 2024-09-16T14:08:04.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3781000000000005e-05 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260697+0000", 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7951000000000002e-05 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89401 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.249522+0000", 2024-09-16T14:08:04.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0176702420000003, 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011292926, 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89401 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249522+0000", 2024-09-16T14:08:04.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249522+0000", 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249523+0000", 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9699999999999995e-07 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249526+0000", 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4400000000000001e-06 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249532+0000", 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1970000000000001e-06 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249536+0000", 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5439999999999999e-06 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260717+0000", 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011180895 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260768+0000", 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0682e-05 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260815+0000", 2024-09-16T14:08:04.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7471e-05 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89402 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.249540+0000", 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0176528539999996, 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011385655999999999, 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89402 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249540+0000", 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249540+0000", 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249540+0000", 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9799999999999999e-07 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249541+0000", 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.426e-06 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249545+0000", 2024-09-16T14:08:04.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3380000000000001e-06 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.249548+0000", 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7129999999999999e-06 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260833+0000", 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011284305999999999 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260882+0000", 2024-09-16T14:08:04.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8930000000000001e-05 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.260925+0000", 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3645e-05 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89425 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250222+0000", 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0169704560000001, 2024-09-16T14:08:04.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011359532, 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89425 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250222+0000", 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250222+0000", 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250222+0000", 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250223+0000", 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2270000000000001e-06 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250229+0000", 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3909999999999997e-06 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250231+0000", 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2809999999999998e-06 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261380+0000", 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011149133 2024-09-16T14:08:04.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261482+0000", 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102157 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261581+0000", 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9056999999999997e-05 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89426 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250233+0000", 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0169590350000002, 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011520753999999999, 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89426 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250233+0000", 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250233+0000", 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250234+0000", 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8599999999999999e-07 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250235+0000", 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.248e-06 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250238+0000", 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1599999999999998e-06 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250241+0000", 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7250000000000002e-06 2024-09-16T14:08:04.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261610+0000", 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011368814 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261687+0000", 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7181999999999994e-05 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261754+0000", 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7339e-05 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89427 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250267+0000", 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0169250630000004, 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011650486999999999, 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89427 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250267+0000", 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250267+0000", 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250268+0000", 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6000000000000003e-07 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250273+0000", 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1760000000000001e-06 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250284+0000", 2024-09-16T14:08:04.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0645999999999999e-05 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250288+0000", 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1049999999999997e-06 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261774+0000", 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011485758 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261846+0000", 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2287999999999996e-05 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261918+0000", 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1853999999999993e-05 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89428 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250292+0000", 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0169000339999998, 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011786985, 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89428 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250292+0000", 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250292+0000", 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250293+0000", 2024-09-16T14:08:04.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6100000000000001e-07 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250295+0000", 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0420000000000001e-06 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250300+0000", 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5659999999999997e-06 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250304+0000", 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5939999999999998e-06 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.261954+0000", 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01165013 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262012+0000", 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7340999999999999e-05 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262079+0000", 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7850999999999998e-05 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89429 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250308+0000", 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0168845549999999, 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011934480000000001, 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.335 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89429 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250308+0000", 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250308+0000", 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250308+0000", 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9299999999999999e-07 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250310+0000", 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8309999999999999e-06 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250315+0000", 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2569999999999998e-06 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250323+0000", 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7379999999999994e-06 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262108+0000", 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011785088000000001 2024-09-16T14:08:04.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262175+0000", 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6555000000000002e-05 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262242+0000", 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.339 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89430 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250328+0000", 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0168647249999996, 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012039619999999999, 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89430 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250328+0000", 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250328+0000", 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250328+0000", 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:08:04.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250330+0000", 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9520000000000001e-06 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250336+0000", 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7910000000000001e-06 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250339+0000", 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262266+0000", 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011926554000000001 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262312+0000", 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6218999999999999e-05 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262367+0000", 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5121999999999998e-05 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89431 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250343+0000", 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0168493569999999, 2024-09-16T14:08:04.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012193994999999999, 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89431 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250343+0000", 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250343+0000", 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250343+0000", 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.01e-07 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250345+0000", 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.778e-06 2024-09-16T14:08:04.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250351+0000", 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.446e-06 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250355+0000", 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3499999999999999e-06 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262395+0000", 2024-09-16T14:08:04.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012040076 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262457+0000", 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1909000000000006e-05 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262537+0000", 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0035000000000002e-05 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89432 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250359+0000", 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0168336379999996, 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012291913, 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89432 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.349 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250359+0000", 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250359+0000", 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250359+0000", 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.89e-07 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250361+0000", 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.793e-06 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250371+0000", 2024-09-16T14:08:04.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9939999999999995e-06 2024-09-16T14:08:04.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250375+0000", 2024-09-16T14:08:04.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1370000000000004e-06 2024-09-16T14:08:04.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262550+0000", 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012174986 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262589+0000", 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9264999999999998e-05 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262651+0000", 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1348999999999998e-05 2024-09-16T14:08:04.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89433 3.b 3:d2ba2f4f:::rbd_data.113618577e70.00000000000000f1:17 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.250379+0000", 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0168133270000004, 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012435178999999999, 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89433 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250379+0000", 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250379+0000", 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250379+0000", 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.84e-07 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250381+0000", 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.871e-06 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250387+0000", 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5849999999999999e-06 2024-09-16T14:08:04.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.250390+0000", 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5190000000000001e-06 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262680+0000", 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012289858000000001 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262759+0000", 2024-09-16T14:08:04.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9103000000000005e-05 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262814+0000", 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4858999999999998e-05 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89450 3.b 3:d6399513:::rbd_data.1136f7114f14.000000000000007b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.254588+0000", 2024-09-16T14:08:04.358 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0126044199999997, 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013066903, 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89450 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254588+0000", 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254588+0000", 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254589+0000", 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4139999999999999e-06 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254737+0000", 2024-09-16T14:08:04.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014749699999999999 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254749+0000", 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1904e-05 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254754+0000", 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5249999999999996e-06 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262826+0000", 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080715230000000006 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.262875+0000", 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9171000000000001e-05 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.263043+0000", 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016784600000000001 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.266058+0000", 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030155889999999999 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.267569+0000", 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015102150000000001 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.267617+0000", 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8328999999999998e-05 2024-09-16T14:08:04.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.267655+0000", 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7889999999999998e-05 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89451 3.b 3:d6399513:::rbd_data.1136f7114f14.000000000000007b:head [delete] snapc 18=[18] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.254760+0000", 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0124320029999998, 2024-09-16T14:08:04.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013757736, 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89451 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254760+0000", 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254760+0000", 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254761+0000", 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254764+0000", 2024-09-16T14:08:04.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.413e-06 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254768+0000", 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7500000000000003e-06 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254772+0000", 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2629999999999999e-06 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:04.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.263916+0000", 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091445490000000001 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.263987+0000", 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0727000000000002e-05 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.264204+0000", 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000217147 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.267361+0000", 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031571910000000002 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.268404+0000", 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001042591 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.268468+0000", 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4455000000000005e-05 2024-09-16T14:08:04.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.268518+0000", 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9945999999999999e-05 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:04.370 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:04.372 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:04.366+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:04.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:04.473 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:04.466+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:04.546 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119198 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540105+0000", 2024-09-16T14:08:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0030276320000000002, 2024-09-16T14:08:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030534780000000001, 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119198 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540105+0000", 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540105+0000", 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540106+0000", 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540113+0000", 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4449999999999999e-06 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540119+0000", 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.083e-06 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540124+0000", 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7949999999999998e-06 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119199 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540128+0000", 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0030047979999999999, 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030915180000000001, 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119199 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540128+0000", 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540128+0000", 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540129+0000", 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540130+0000", 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9209999999999999e-06 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540136+0000", 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0059999999999998e-06 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540140+0000", 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8020000000000002e-06 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119204 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540145+0000", 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0029876299999999998, 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031372990000000001, 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119204 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540145+0000", 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540145+0000", 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540146+0000", 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4700000000000002e-07 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540148+0000", 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9769999999999999e-06 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540153+0000", 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5210000000000002e-06 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540157+0000", 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7629999999999998e-06 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119205 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540214+0000", 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.002919086, 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031238030000000001, 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119205 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540214+0000", 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540214+0000", 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540214+0000", 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5599999999999995e-07 2024-09-16T14:08:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540218+0000", 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999999e-06 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540378+0000", 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016025699999999999 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540383+0000", 2024-09-16T14:08:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.215e-06 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119210 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540389+0000", 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.002743554, 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029987249999999998, 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119210 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540389+0000", 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540389+0000", 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540390+0000", 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5499999999999998e-07 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540393+0000", 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4039999999999999e-06 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540400+0000", 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6259999999999997e-06 2024-09-16T14:08:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540404+0000", 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119211 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540408+0000", 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0027248390000000002, 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030356509999999999, 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119211 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540408+0000", 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540408+0000", 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540408+0000", 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540411+0000", 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540416+0000", 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7389999999999996e-06 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540420+0000", 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6540000000000001e-06 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119212 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540469+0000", 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0026642189999999998, 2024-09-16T14:08:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030290120000000002, 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119212 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540469+0000", 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540469+0000", 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540469+0000", 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7400000000000003e-07 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540472+0000", 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2140000000000001e-06 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540480+0000", 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9170000000000006e-06 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540485+0000", 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9439999999999996e-06 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119213 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540490+0000", 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0026429050000000001, 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030596960000000002, 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119213 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540490+0000", 2024-09-16T14:08:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540490+0000", 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540490+0000", 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540492+0000", 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0540000000000002e-06 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540499+0000", 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5830000000000003e-06 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540503+0000", 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9659999999999998e-06 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119219 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540507+0000", 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0026255760000000001, 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030939129999999998, 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119219 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540507+0000", 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540507+0000", 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540508+0000", 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540510+0000", 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0439999999999998e-06 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540516+0000", 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6279999999999999e-06 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540520+0000", 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8909999999999997e-06 2024-09-16T14:08:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119221 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540524+0000", 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0026085919999999999, 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031317210000000001, 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119221 2024-09-16T14:08:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540524+0000", 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540524+0000", 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540525+0000", 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540527+0000", 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9989999999999998e-06 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540534+0000", 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.272e-06 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540538+0000", 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6260000000000001e-06 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119222 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540542+0000", 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0025904819999999999, 2024-09-16T14:08:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031668070000000002, 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119222 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540542+0000", 2024-09-16T14:08:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540542+0000", 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540543+0000", 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2099999999999998e-07 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540545+0000", 2024-09-16T14:08:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9120000000000001e-06 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540826+0000", 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028091200000000002 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540831+0000", 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7209999999999999e-06 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119224 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540839+0000", 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.002294304, 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002924659, 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119224 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540839+0000", 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540839+0000", 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540839+0000", 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3900000000000004e-07 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540843+0000", 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.568e-06 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540850+0000", 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4120000000000004e-06 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540901+0000", 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1224000000000003e-05 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119231 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540908+0000", 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.002225103, 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002907944, 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119231 2024-09-16T14:08:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540908+0000", 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540908+0000", 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540908+0000", 2024-09-16T14:08:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540929+0000", 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0310999999999999e-05 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.542145+0000", 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001216747 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.542150+0000", 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8840000000000002e-06 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119232 3.13 3.5fee67f3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.542157+0000", 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00097600699999999998, 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001710601, 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119232 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.542157+0000", 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.542157+0000", 2024-09-16T14:08:04.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.542158+0000", 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6000000000000003e-07 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.542161+0000", 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6880000000000001e-06 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.542168+0000", 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9229999999999996e-06 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.542172+0000", 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4809999999999998e-06 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119251 3.c 3:317dbf57:::rbd_data.113681563d6c.000000000000000c:15 [read 421888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543450+0000", 2024-09-16T14:08:04.592 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.9996834, 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00047542, 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119251 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543450+0000", 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543450+0000", 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543450+0000", 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5000000000000002e-07 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543453+0000", 2024-09-16T14:08:04.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6170000000000001e-06 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543459+0000", 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1720000000000004e-06 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543463+0000", 2024-09-16T14:08:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8489999999999999e-06 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543894+0000", 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00043073800000000001 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119252 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.596 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543467+0000", 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.9996662, 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00052164400000000004, 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119252 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543467+0000", 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543467+0000", 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543467+0000", 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6800000000000001e-07 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543469+0000", 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0810000000000001e-06 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543475+0000", 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7579999999999998e-06 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543479+0000", 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119257 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.599 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543483+0000", 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.99965, 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000560265, 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119257 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543483+0000", 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543483+0000", 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543483+0000", 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7500000000000001e-07 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543502+0000", 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8893999999999999e-05 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543510+0000", 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.486e-06 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543514+0000", 2024-09-16T14:08:04.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4190000000000004e-06 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119258 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543520+0000", 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.9996133, 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00057539400000000001, 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.603 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119258 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543520+0000", 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543520+0000", 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543520+0000", 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6499999999999999e-07 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543522+0000", 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0470000000000001e-06 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543528+0000", 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0599999999999996e-06 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543532+0000", 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8979999999999998e-06 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119261 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543536+0000", 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.9995966, 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00061594999999999998, 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119261 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543536+0000", 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543536+0000", 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543537+0000", 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0499999999999999e-07 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543539+0000", 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3250000000000002e-06 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543585+0000", 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6026999999999998e-05 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543591+0000", 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8300000000000001e-06 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119262 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543597+0000", 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.999536, 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00060786300000000004, 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.609 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119262 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543597+0000", 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543597+0000", 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543598+0000", 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.85e-07 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543601+0000", 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8370000000000001e-06 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543608+0000", 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4760000000000001e-06 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543612+0000", 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1359999999999999e-06 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119263 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543616+0000", 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.9995165, 2024-09-16T14:08:04.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00064056300000000003, 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119263 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543616+0000", 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543616+0000", 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543617+0000", 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.46e-07 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543619+0000", 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2139999999999999e-06 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543625+0000", 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9909999999999999e-06 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543629+0000", 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6339999999999999e-06 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119264 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543633+0000", 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.9995003, 2024-09-16T14:08:04.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067763600000000002, 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119264 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543633+0000", 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543633+0000", 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543633+0000", 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6400000000000003e-07 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543635+0000", 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0279999999999999e-06 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543641+0000", 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0009999999999998e-06 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543645+0000", 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119196 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 302080~1024 out=1024b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.540072+0000", 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0030611789999999998, 2024-09-16T14:08:04.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043057269999999996, 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119196 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540072+0000", 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540072+0000", 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540072+0000", 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.89e-07 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540075+0000", 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9579999999999999e-06 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540082+0000", 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5239999999999997e-06 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540085+0000", 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8130000000000002e-06 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540586+0000", 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050056299999999998 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.540708+0000", 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121659 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119269 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543650+0000", 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.9994831, 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00079281499999999999, 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119269 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543650+0000", 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543650+0000", 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543650+0000", 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.32e-07 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543652+0000", 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2960000000000001e-06 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543658+0000", 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8130000000000001e-06 2024-09-16T14:08:04.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543662+0000", 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6210000000000001e-06 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:119271 3.c 3.eafdbe8c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:04.543666+0000", 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4294967295.9994669, 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082936200000000005, 2024-09-16T14:08:04.625 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 119271 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543666+0000", 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543666+0000", 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543666+0000", 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4799999999999999e-07 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543668+0000", 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.153e-06 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543674+0000", 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8320000000000002e-06 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:04.543678+0000", 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5149999999999998e-06 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 25 2024-09-16T14:08:04.628 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:04.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:04.633 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:04.625+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:04.731 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:04.725+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:04.732 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:04.732 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:04.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:04.747 INFO:teuthology.orchestra.run.smithi079.stdout:2628 write 0xaf363d2 thru 0xaf3c6f8 (0x6327 bytes) 2024-09-16T14:08:04.753 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:04.753 INFO:teuthology.orchestra.run.smithi079.stdout:2631 writesame 0x9431159 thru 0x943b39f data_size 0x1(0xa247 bytes) 2024-09-16T14:08:04.788 INFO:teuthology.orchestra.run.smithi079.stdout:2633 write 0xa3707d2 thru 0xa37cac4 (0xc2f3 bytes) 2024-09-16T14:08:04.810 INFO:teuthology.orchestra.run.smithi079.stdout:2636 write 0xdef643e thru 0xdefe6bd (0x8280 bytes) 2024-09-16T14:08:04.815 INFO:teuthology.orchestra.run.smithi079.stdout:2637 writesame 0x2ce5a67 thru 0x2cf4835 data_size 0x15(0xedcf bytes) 2024-09-16T14:08:04.829 INFO:teuthology.orchestra.run.smithi079.stdout:2639 trunc from 0xdefe6be to 0xc2b3b2c 2024-09-16T14:08:04.832 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:04.825+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:04.840 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.928031021000001, 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.928021294000001, 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927997731, 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927970492, 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927961078999999, 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.92793676, 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927917153999999, 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927896966, 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927873697000001, 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927843447000001, 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927826695, 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927809431, 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927770595, 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927735767, 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927699881000001, 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927677289, 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927648217, 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927629871000001, 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927612531999999, 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.927596609, 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:04.946 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:04.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:04.948 INFO:teuthology.orchestra.run.smithi079.stdout:2640 writesame 0x83cf30a thru 0x83d6e57 data_size 0x3(0x7b4e bytes) 2024-09-16T14:08:04.948 INFO:teuthology.orchestra.run.smithi079.stdout:2643 write 0xd252659 thru 0xd25da5a (0xb402 bytes) 2024-09-16T14:08:04.948 INFO:teuthology.orchestra.run.smithi079.stdout:2644 write 0x5aa8655 thru 0x5ab24aa (0x9e56 bytes) 2024-09-16T14:08:04.949 INFO:teuthology.orchestra.run.smithi079.stdout:2645 write 0x7982a93 thru 0x7983e78 (0x13e6 bytes) 2024-09-16T14:08:04.949 INFO:teuthology.orchestra.run.smithi079.stdout:2646 punch from 0xca2d2c6 to 0xca37c8a, (0xa9c4 bytes) 2024-09-16T14:08:04.949 INFO:teuthology.orchestra.run.smithi079.stdout:2648 punch from 0x7f7b8d8 to 0x7f8ae1a, (0xf542 bytes) 2024-09-16T14:08:04.949 INFO:teuthology.orchestra.run.smithi079.stdout:2649 trunc from 0xd25da5b to 0x74b0d88 2024-09-16T14:08:04.949 INFO:teuthology.orchestra.run.smithi079.stdout:2650 trunc from 0x74b0d88 to 0x45d5124 2024-09-16T14:08:04.949 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:04.942+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:04.979 INFO:teuthology.orchestra.run.smithi079.stdout:2652 write 0x7d2194e thru 0x7d2dbf2 (0xc2a5 bytes) 2024-09-16T14:08:05.010 INFO:teuthology.orchestra.run.smithi079.stdout:2653 writesame 0xc6b30c6 thru 0xc6b3405 data_size 0x2(0x340 bytes) 2024-09-16T14:08:05.031 INFO:teuthology.orchestra.run.smithi079.stdout:2654 trunc from 0xc6b3406 to 0x1b7ddec 2024-09-16T14:08:05.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:05.049 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:05.042+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:05.061 INFO:teuthology.orchestra.run.smithi079.stdout:2655 read 0x1502e7f thru 0x150798b (0x4b0d bytes) 2024-09-16T14:08:05.062 INFO:teuthology.orchestra.run.smithi079.stdout:2656 read 0x14f23df thru 0x14f39d2 (0x15f4 bytes) 2024-09-16T14:08:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:2657 read 0x92b9be thru 0x93a4b2 (0xeaf5 bytes) 2024-09-16T14:08:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:05.093 INFO:teuthology.orchestra.run.smithi079.stdout:2658 write 0xd182b05 thru 0xd186d34 (0x4230 bytes) 2024-09-16T14:08:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:2659 read 0x75a8f16 thru 0x75acd92 (0x3e7d bytes) 2024-09-16T14:08:05.099 INFO:teuthology.orchestra.run.smithi079.stdout:2660 writesame 0xa3567f6 thru 0xa357291 data_size 0x7(0xa9c bytes) 2024-09-16T14:08:05.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:05.105 INFO:teuthology.orchestra.run.smithi079.stdout:2661 read 0x5d3b10a thru 0x5d44e5a (0x9d51 bytes) 2024-09-16T14:08:05.108 INFO:teuthology.orchestra.run.smithi079.stdout:2662 trunc from 0xd186d35 to 0xee008c6 2024-09-16T14:08:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:2664 read 0xe1a2dca thru 0xe1ae18f (0xb3c6 bytes) 2024-09-16T14:08:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:2668 punch from 0xb5ece6c to 0xb5f06e6, (0x387a bytes) 2024-09-16T14:08:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:2669 punch from 0xbee1d37 to 0xbee580b, (0x3ad4 bytes) 2024-09-16T14:08:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:2671 write 0x316cd81 thru 0x3176b9c (0x9e1c bytes) 2024-09-16T14:08:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:2672 punch from 0x116b68e to 0x1177736, (0xc0a8 bytes) 2024-09-16T14:08:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:05.124 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:05.125 INFO:teuthology.orchestra.run.smithi079.stdout:2676 write 0xac72e30 thru 0xac7a2f9 (0x74ca bytes) 2024-09-16T14:08:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:2677 clone 23 order 21 su 16384 sc 15 2024-09-16T14:08:05.150 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:05.145+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:05.176 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:05.196 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:05.196 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:05.197 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:05.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:05.217 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:05.245 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:05.253 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:05.246+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:05.278 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:05.311 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:05.339 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:05.339 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:05.343 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:05.343 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:05.343 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:05.344 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:05.344 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:05.344 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:05.344 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:05.344 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:05.344 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:05.346 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:05.346 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:05.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:05.352 INFO:teuthology.orchestra.run.smithi079.stderr:Nothing to do for non-filestore backend 2024-09-16T14:08:05.353 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:05.346+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:05.455 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:05.450+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:05.493 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:05.494 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:05.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:05.556 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:05.549+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:05.604 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:05.604 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:05.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:05.658 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:05.653+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:05.713 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:05.713 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:05.722 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:05.767 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:05.762+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:05.828 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:05.828 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:05.838 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:05.858 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:05.853+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:05.889 DEBUG:teuthology.orchestra.run.smithi079:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op apply-layout-settings --pool rbd 2024-09-16T14:08:05.937 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:05.938 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:05.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:05.991 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:05.986+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:06.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:08:06.050 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:06.050 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:06.058 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:06.054+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:06.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:06.160 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:06.153+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:06.162 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:06.162 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:06.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:06.260 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:06.253+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:06.271 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone22 from 0xee008c6 (overlap 0x1b7ddec) to 0x18be154 2024-09-16T14:08:06.271 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:06.271 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:06.280 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:06.362 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:06.354+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:06.378 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:06.378 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:06.387 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:06.462 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:06.457+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:06.536 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:06.536 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:06.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:06.563 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:06.558+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:06.640 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:06.640 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:06.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:06.664 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:06.657+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:06.711 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-16T14:08:06.738 INFO:teuthology.orchestra.run.smithi079.stdout:2678 read 0x4bee467 thru 0x4bf8f26 (0xaac0 bytes) 2024-09-16T14:08:06.745 INFO:teuthology.orchestra.run.smithi079.stdout:2679 write 0x459b799 thru 0x45a94ab (0xdd13 bytes) 2024-09-16T14:08:06.762 INFO:teuthology.orchestra.run.smithi079.stdout:2680 write 0x7956822 thru 0x795a207 (0x39e6 bytes) 2024-09-16T14:08:06.765 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:06.758+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:06.771 INFO:teuthology.orchestra.run.smithi079.stdout:2681 read 0x1ae667f thru 0x1af3f04 (0xd886 bytes) 2024-09-16T14:08:06.779 INFO:teuthology.orchestra.run.smithi079.stdout:2682 punch from 0x634b7ac to 0x6353442, (0x7c96 bytes) 2024-09-16T14:08:06.782 INFO:teuthology.orchestra.run.smithi079.stdout:2684 writesame 0x41d786f thru 0x41dae08 data_size 0x3(0x359a bytes) 2024-09-16T14:08:06.798 INFO:teuthology.orchestra.run.smithi079.stdout:2686 write 0x543bc2a thru 0x5441c7e (0x6055 bytes) 2024-09-16T14:08:06.807 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:06.807 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:06.808 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:06.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:06.818 INFO:teuthology.orchestra.run.smithi079.stdout:2688 trunc from 0xee008c6 to 0xa808ff5 2024-09-16T14:08:06.866 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:06.858+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:06.911 INFO:teuthology.orchestra.run.smithi079.stdout:2690 punch from 0x3367940 to 0x33728f3, (0xafb3 bytes) 2024-09-16T14:08:06.911 INFO:teuthology.orchestra.run.smithi079.stdout:2691 writesame 0x4037493 thru 0x4044086 data_size 0x13(0xcbf4 bytes) 2024-09-16T14:08:06.917 INFO:teuthology.orchestra.run.smithi079.stdout:2692 read 0x7030c5b thru 0x70341e2 (0x3588 bytes) 2024-09-16T14:08:06.917 INFO:teuthology.orchestra.run.smithi079.stdout:2693 punch from 0x59ae521 to 0x59bdb41, (0xf620 bytes) 2024-09-16T14:08:06.917 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:06.918 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:06.927 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:06.931 INFO:teuthology.orchestra.run.smithi079.stdout:2694 trunc from 0xa808ff5 to 0x2d8db6e 2024-09-16T14:08:06.971 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:06.966+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:06.994 INFO:teuthology.orchestra.run.smithi079.stdout:2695 read 0x16b1b16 thru 0x16bad57 (0x9242 bytes) 2024-09-16T14:08:07.018 INFO:teuthology.orchestra.run.smithi079.stdout:2698 writesame 0xa7307ca thru 0xa7319ed data_size 0x2b(0x1224 bytes) 2024-09-16T14:08:07.018 INFO:teuthology.orchestra.run.smithi079.stdout:2699 punch from 0x3f6ae68 to 0x3f7a60f, (0xf7a7 bytes) 2024-09-16T14:08:07.018 INFO:teuthology.orchestra.run.smithi079.stdout:2700 read 0x290b813 thru 0x291b4da (0xfcc8 bytes) 2024-09-16T14:08:07.021 INFO:teuthology.orchestra.run.smithi079.stdout:2701 read 0x8297c84 thru 0x8299ea5 (0x2222 bytes) 2024-09-16T14:08:07.021 INFO:teuthology.orchestra.run.smithi079.stdout:2702 read 0x4c7c600 thru 0x4c7f3e5 (0x2de6 bytes) 2024-09-16T14:08:07.021 INFO:teuthology.orchestra.run.smithi079.stdout:2703 read 0xa5a3336 thru 0xa5aab53 (0x781e bytes) 2024-09-16T14:08:07.021 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:07.021 INFO:teuthology.orchestra.run.smithi079.stdout:2707 read 0x8bfc9f1 thru 0x8c0a3a9 (0xd9b9 bytes) 2024-09-16T14:08:07.021 INFO:teuthology.orchestra.run.smithi079.stdout:2710 read 0x9ae3086 thru 0x9af2177 (0xf0f2 bytes) 2024-09-16T14:08:07.022 INFO:teuthology.orchestra.run.smithi079.stdout:2711 read 0x7f6f5bd thru 0x7f740a6 (0x4aea bytes) 2024-09-16T14:08:07.022 INFO:teuthology.orchestra.run.smithi079.stdout:2712 read 0x3958478 thru 0x3959b55 (0x16de bytes) 2024-09-16T14:08:07.022 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:07.022 INFO:teuthology.orchestra.run.smithi079.stdout:2714 read 0xad4670 thru 0xae3d28 (0xf6b9 bytes) 2024-09-16T14:08:07.029 INFO:teuthology.orchestra.run.smithi079.stdout:2715 clone 24 order 21 su 262144 sc 5 2024-09-16T14:08:07.041 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:08:07.051 INFO:teuthology.orchestra.run.smithi079.stderr:Nothing to do for non-filestore backend 2024-09-16T14:08:07.068 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:07.062+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:07.080 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:07.080 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:07.089 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:07.169 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:07.162+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:07.183 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:07.183 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:07.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:07.270 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:07.265+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:07.304 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:07.305 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:07.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:07.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:07.371 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:07.365+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:07.411 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:07.411 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:07.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:07.472 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:07.465+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:07.498 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:07.518 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:07.518 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:07.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:07.518 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:07.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.518 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.518 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604990832, 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:07.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604981105, 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:07.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:07.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604957541999999, 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.534 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604930303, 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:07.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:07.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:07.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604920890000001, 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:07.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:07.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.545 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604896570999999, 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:07.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:07.550 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604876965000001, 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:07.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.556 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604856777, 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:07.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604833508, 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:07.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:07.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:07.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604803258, 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:07.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:07.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:07.572 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604786506, 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:07.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:07.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604769242, 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:07.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:07.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604730406, 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:07.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:07.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604695577999999, 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:07.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:07.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:07.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604659692, 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:07.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.598 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.6046371, 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.599 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604608027999999, 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.604 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:07.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604589682, 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:07.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:07.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:07.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:07.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:07.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:07.615 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.604572342999999, 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:07.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.60455642, 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.621 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:07.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.626 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.626 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.626 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.626 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.626 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:07.627 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:07.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:07.628 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:07.628 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:07.629 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:07.629 DEBUG:teuthology.orchestra.run.smithi079:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op apply-layout-settings --pool pool_client.0 2024-09-16T14:08:07.630 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:07.630 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:07.622+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:07.640 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:07.640 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:07.728 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:07.721+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:07.735 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:07.735 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:07.735 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:07.735 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:07.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:07.829 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:07.822+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:07.840 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:07.840 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:07.840 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:07.841 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:07.841 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:07.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:07.930 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:07.926+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:07.954 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.954 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.046439693, 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.955 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:07.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:08:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:08:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:08:07.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:08:07.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.046416985, 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:07.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:08:07.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.967 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.046392833000001, 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:08:07.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.046359892, 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:08:07.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.978 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.046335209, 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:07.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:08:07.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:08:07.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.04631198, 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:07.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:08:07.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.044466015999999, 2024-09-16T14:08:07.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:08:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:08:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.992 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:07.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:08:07.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:07.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:08:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:08:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:08:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:08:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.044423152, 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:07.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:08:08.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042963534, 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:08:08.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:08:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:08:08.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042947764999999, 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:08:08.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:08:08.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:08:08.014 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042932013, 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.015 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:08:08.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:08:08.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042916582, 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:08:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:08:08.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:08:08.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:08:08.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:08:08.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:08:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:08:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:08:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.029 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042900943999999, 2024-09-16T14:08:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:08:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:08:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:08:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:08:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:08:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042885804999999, 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:08:08.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:08:08.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.042 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042870968000001, 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:08.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042847964, 2024-09-16T14:08:08.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:08:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:08:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:08:08.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042832739, 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:08:08.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:08:08.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.060 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042817554999999, 2024-09-16T14:08:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:08:08.061 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:08:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:08:08.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042802310000001, 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:08:08.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:08:08.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.042787152000001, 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:08.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:08:08.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.078 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.079 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.079 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:08.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:08.081 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:08.074+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:08.181 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:08.174+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:08.182 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:08.182 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:08.182 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:08.182 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:08.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:08.282 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:08.273+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:08.347 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:08.348 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:08.348 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:08.348 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:08.348 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:08.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:08.349 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone23 from 0xa7319ee (overlap 0x2d8db6e) to 0x1b55075 2024-09-16T14:08:08.383 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:08.378+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:08.467 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86627 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 115712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2903242769999999, 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057990535000000003, 2024-09-16T14:08:08.468 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86627 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.79e-07 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172073+0000", 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.53e-06 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172077+0000", 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172079+0000", 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:08.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224325+0000", 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052246133 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224394+0000", 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8960000000000004e-05 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.230062+0000", 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056682820000000002 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86989 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2687154310000004, 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058737710999999998, 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.473 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86989 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:08.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193681+0000", 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193682+0000", 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6169999999999999e-06 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193687+0000", 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7210000000000002e-06 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193690+0000", 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.072e-06 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.246881+0000", 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.053191117000000003 2024-09-16T14:08:08.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252309+0000", 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054275979999999996 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252418+0000", 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109255 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86995 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2686892289999996, 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059640662999999997, 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86995 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197498+0000", 2024-09-16T14:08:08.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037915219999999999 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197504+0000", 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7520000000000002e-06 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197515+0000", 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.117e-05 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197521+0000", 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.083e-06 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253121+0000", 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055600254000000002 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253227+0000", 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010607000000000001 2024-09-16T14:08:08.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253347+0000", 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119812 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87017 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2647534930000006, 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676182999999998, 2024-09-16T14:08:08.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87017 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197643+0000", 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197645+0000", 2024-09-16T14:08:08.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197652+0000", 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8430000000000004e-06 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197656+0000", 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254046+0000", 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056390033999999999 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254205+0000", 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158639 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254318+0000", 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113916 2024-09-16T14:08:08.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87019 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2647359169999994, 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056580459, 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87019 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197663+0000", 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.396e-06 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197669+0000", 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4099999999999996e-06 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197673+0000", 2024-09-16T14:08:08.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254066+0000", 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056392870999999997 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254160+0000", 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4574000000000006e-05 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254240+0000", 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0125999999999998e-05 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.492 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87023 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 193536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2647190500000001, 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056736048999999997, 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87023 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.493 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197679+0000", 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-16T14:08:08.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197686+0000", 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197690+0000", 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.177e-06 2024-09-16T14:08:08.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254263+0000", 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056573364000000001 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254334+0000", 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0574000000000006e-05 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254413+0000", 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079999999999995e-05 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.498 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87027 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2647019270000008, 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056974745, 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.499 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87027 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2599999999999998e-07 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197696+0000", 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1679999999999998e-06 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197703+0000", 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4389999999999997e-06 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197707+0000", 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254383+0000", 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676509999999999 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254483+0000", 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9692999999999996e-05 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254669+0000", 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018580300000000001 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87029 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 197632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2646850900000004, 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057332283999999997, 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87029 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197713+0000", 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.199e-06 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197719+0000", 2024-09-16T14:08:08.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.699e-06 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197723+0000", 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000002e-06 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254709+0000", 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056986536999999997 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254790+0000", 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0192999999999996e-05 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255043+0000", 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025339500000000001 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.509 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87031 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 234496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2646687980000006, 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057120535, 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87031 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0600000000000001e-07 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197729+0000", 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0499999999999999e-06 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197736+0000", 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2249999999999997e-06 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197739+0000", 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6839999999999998e-06 2024-09-16T14:08:08.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254697+0000", 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056957902999999997 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254776+0000", 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254848+0000", 2024-09-16T14:08:08.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1044999999999995e-05 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87035 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 198656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2646527939999999, 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057340290000000002, 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87035 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:08.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197744+0000", 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6499999999999999e-07 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197746+0000", 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0169999999999999e-06 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197752+0000", 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1040000000000003e-06 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197755+0000", 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.782e-06 2024-09-16T14:08:08.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254869+0000", 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057113766000000003 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254971+0000", 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101932 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255083+0000", 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112124 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87039 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 238592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2646363839999992, 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057586228000000003, 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87039 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197760+0000", 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197762+0000", 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.829e-06 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197768+0000", 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5470000000000002e-06 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197772+0000", 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4699999999999998e-06 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255109+0000", 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057337232000000002 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255184+0000", 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5593000000000006e-05 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255346+0000", 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016120899999999999 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87041 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 202752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2646203299999996, 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057827176000000001, 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87041 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:08.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197776+0000", 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197778+0000", 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3709999999999998e-06 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197784+0000", 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.054e-06 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197787+0000", 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1879999999999998e-06 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255397+0000", 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057609059999999997 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255483+0000", 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6415999999999994e-05 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255603+0000", 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119703 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87043 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 203776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.529 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:08.529 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2646041629999996, 2024-09-16T14:08:08.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057723202000000001, 2024-09-16T14:08:08.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.529 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87043 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197801+0000", 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5869999999999993e-06 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197807+0000", 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197811+0000", 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0099999999999997e-06 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255377+0000", 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057566063000000001 2024-09-16T14:08:08.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255450+0000", 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3192e-05 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255515+0000", 2024-09-16T14:08:08.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4543999999999997e-05 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87045 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2645799699999998, 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057864405000000001, 2024-09-16T14:08:08.537 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87045 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:08.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197819+0000", 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.311e-06 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197824+0000", 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6980000000000003e-06 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197828+0000", 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2339999999999999e-06 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255537+0000", 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057709979000000002 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255613+0000", 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5807000000000001e-05 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255680+0000", 2024-09-16T14:08:08.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6935999999999997e-05 2024-09-16T14:08:08.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.543 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.543 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87051 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2645648180000002, 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058010645999999999, 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.544 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87051 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200527+0000", 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026964990000000002 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200532+0000", 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8899999999999998e-06 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200541+0000", 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1249999999999993e-06 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200546+0000", 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4129999999999997e-06 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255630+0000", 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055083612999999997 2024-09-16T14:08:08.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255711+0000", 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1934000000000005e-05 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255842+0000", 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013017200000000001 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87559 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 100352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2617304619999992, 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056509688000000002, 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87559 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.549 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200666+0000", 2024-09-16T14:08:08.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200668+0000", 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.21e-06 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200674+0000", 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.896e-06 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200678+0000", 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7050000000000001e-06 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.256898+0000", 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056220883999999999 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257014+0000", 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011530499999999999 2024-09-16T14:08:08.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257175+0000", 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016127200000000001 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87568 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:08.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.261698483, 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056739408999999998, 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87568 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200698+0000", 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200700+0000", 2024-09-16T14:08:08.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.272e-06 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200706+0000", 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7080000000000002e-06 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200709+0000", 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.394e-06 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257221+0000", 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056512383999999999 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257319+0000", 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7510000000000007e-05 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257437+0000", 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011773399999999999 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87570 3.e 3:73b2712a:::rbd_data.113619b3ee73.0000000000000027:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2616831509999997, 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056717201000000002, 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.558 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87570 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201471+0000", 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075828899999999999 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201480+0000", 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.448e-06 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201494+0000", 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3162999999999999e-05 2024-09-16T14:08:08.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201500+0000", 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0090000000000004e-06 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257205+0000", 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055705603999999999 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257330+0000", 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124972 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257430+0000", 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9716000000000005e-05 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87651 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 136192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2567834449999999, 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056431633000000002, 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87651 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205613+0000", 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999998e-07 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205614+0000", 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:08.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205619+0000", 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.758e-06 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205622+0000", 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0690000000000002e-06 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.261887+0000", 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056265241000000001 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.261965+0000", 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7671999999999998e-05 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262044+0000", 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9320999999999995e-05 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87773 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 63488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:08.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2559061370000002, 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056458201, 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87773 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:08.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:08.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207001+0000", 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051104000000000004 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207006+0000", 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207019+0000", 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2266000000000001e-05 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207025+0000", 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2029999999999997e-06 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262711+0000", 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055686539 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262801+0000", 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9895999999999994e-05 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:08.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262948+0000", 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146642 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:08.574 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:08.576 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:08.577 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:08.570+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:08.677 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:08.670+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:08.684 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:08.685 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:08.685 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:08.685 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:08.695 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:08.770 INFO:teuthology.orchestra.run.smithi079.stderr:Nothing to do for non-filestore backend 2024-09-16T14:08:08.777 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:08.770+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:08.797 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:08.797 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:08.798 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:08.798 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:08.798 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:08.807 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:08.878 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:08.873+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1143 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.635695939000001, 2024-09-16T14:08:08.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018939362000000001, 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1143 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:08.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.44e-07 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336884+0000", 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336891+0000", 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336895+0000", 2024-09-16T14:08:08.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0820000000000001e-06 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349584+0000", 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012689283000000001 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349708+0000", 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123793 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350151+0000", 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044259900000000001 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352993+0000", 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028426979999999998 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355720+0000", 2024-09-16T14:08:08.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002726642 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355762+0000", 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2116000000000001e-05 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355820+0000", 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8118999999999997e-05 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.635677833000003, 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.983 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:08.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:08.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.635655436, 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:08.989 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:08.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:08.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:08.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:08.995 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.635048724999997, 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:08.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:08.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:08.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.001 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.634974397999997, 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:09.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:09.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:09.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.664049605999999, 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:09.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.036091024999998, 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:09.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:09.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:09.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:09.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.505930558999999, 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:09.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:09.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:09.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.8583820279999994, 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:09.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.029 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101781 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3121362349999997, 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019058275, 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101781 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:08:09.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660443+0000", 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660450+0000", 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999997e-06 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660454+0000", 2024-09-16T14:08:09.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0029999999999996e-06 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679188+0000", 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018734869000000001 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679271+0000", 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2721999999999998e-05 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679499+0000", 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227612 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101783 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3121190570000003, 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019291800000000001, 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.034 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101783 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4499999999999997e-07 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660465+0000", 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4659999999999996e-06 2024-09-16T14:08:09.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660471+0000", 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5250000000000002e-06 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660480+0000", 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5779999999999999e-06 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679530+0000", 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019050085000000001 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679633+0000", 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103341 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679750+0000", 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011636 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.312091788, 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:09.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.043 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3120745969999996, 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:09.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:09.048 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3120375800000001, 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:09.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:09.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.312016904, 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:09.053 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.057 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3119747239999997, 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:09.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:09.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3119404990000003, 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.062 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:09.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:09.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3119107830000001, 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:09.066 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.070 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3118911210000004, 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:09.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:09.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:09.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3031309960000002, 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.076 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:09.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.080 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:09.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:08:09.083 INFO:teuthology.orchestra.run.smithi079.stdout:2716 trunc from 0xa7319ee to 0x67e95cf 2024-09-16T14:08:09.083 INFO:teuthology.orchestra.run.smithi079.stdout:2717 trunc from 0x67e95cf to 0x536b381 2024-09-16T14:08:09.083 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:09.084 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:09.074+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:09.111 INFO:teuthology.orchestra.run.smithi079.stdout:2719 writesame 0x9b54290 thru 0x9b55164 data_size 0x1(0xed5 bytes) 2024-09-16T14:08:09.111 INFO:teuthology.orchestra.run.smithi079.stdout:2720 write 0x7cd5f2c thru 0x7cdedcc (0x8ea1 bytes) 2024-09-16T14:08:09.118 INFO:teuthology.orchestra.run.smithi079.stdout:2721 write 0x967b58f thru 0x968b07b (0xfaed bytes) 2024-09-16T14:08:09.127 INFO:teuthology.orchestra.run.smithi079.stdout:2722 write 0x9a91da0 thru 0x9a92638 (0x899 bytes) 2024-09-16T14:08:09.133 INFO:teuthology.orchestra.run.smithi079.stdout:2723 punch from 0x3a2bcac to 0x3a37b6e, (0xbec2 bytes) 2024-09-16T14:08:09.133 INFO:teuthology.orchestra.run.smithi079.stdout:2724 read 0x5dc538a thru 0x5dd4382 (0xeff9 bytes) 2024-09-16T14:08:09.133 INFO:teuthology.orchestra.run.smithi079.stdout:2725 writesame 0x4b991ae thru 0x4ba7ed5 data_size 0x4(0xed28 bytes) 2024-09-16T14:08:09.152 INFO:teuthology.orchestra.run.smithi079.stdout:2728 punch from 0x281aa97 to 0x28221ba, (0x7723 bytes) 2024-09-16T14:08:09.169 INFO:teuthology.orchestra.run.smithi079.stdout:2730 write 0xc75e3d7 thru 0xc7656a2 (0x72cc bytes) 2024-09-16T14:08:09.176 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:09.176 INFO:teuthology.orchestra.run.smithi079.stdout:2734 read 0x30e6533 thru 0x30e6f4f (0xa1d bytes) 2024-09-16T14:08:09.178 INFO:teuthology.orchestra.run.smithi079.stdout:2735 punch from 0x844e9b3 to 0x8457f5e, (0x95ab bytes) 2024-09-16T14:08:09.182 INFO:teuthology.orchestra.run.smithi079.stdout:2736 punch from 0xc1706a4 to 0xc172744, (0x20a0 bytes) 2024-09-16T14:08:09.182 INFO:teuthology.orchestra.run.smithi079.stdout:2737 read 0x561d228 thru 0x5626eb0 (0x9c89 bytes) 2024-09-16T14:08:09.183 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:09.183 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:09.185 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:09.181+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:09.196 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:09.196 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:09.197 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:09.198 INFO:teuthology.orchestra.run.smithi079.stdout:2740 trunc from 0xc7656a3 to 0x40e2605 2024-09-16T14:08:09.208 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:09.243 INFO:teuthology.orchestra.run.smithi079.stdout:2741 writesame 0x6700eba thru 0x670dc33 data_size 0x21(0xcd7a bytes) 2024-09-16T14:08:09.256 INFO:teuthology.orchestra.run.smithi079.stdout:2742 write 0x70297ed thru 0x7034915 (0xb129 bytes) 2024-09-16T14:08:09.264 INFO:teuthology.orchestra.run.smithi079.stdout:2743 trunc from 0x7034916 to 0x63b6b3f 2024-09-16T14:08:09.283 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:09.278+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:09.294 INFO:teuthology.orchestra.run.smithi079.stdout:2744 trunc from 0x63b6b3f to 0x2989084 2024-09-16T14:08:09.311 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:09.311 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:09.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:09.357 INFO:teuthology.orchestra.run.smithi079.stdout:2745 read 0x195939e thru 0x195a519 (0x117c bytes) 2024-09-16T14:08:09.360 INFO:teuthology.orchestra.run.smithi079.stdout:2746 punch from 0x1a8250f to 0x1a88a21, (0x6512 bytes) 2024-09-16T14:08:09.366 INFO:teuthology.orchestra.run.smithi079.stdout:2747 trunc from 0x2989084 to 0x5128a61 2024-09-16T14:08:09.384 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:09.378+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:09.419 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:09.419 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102411 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7440580040000002, 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050639235999999997, 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102411 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0499999999999999e-07 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670939+0000", 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8409999999999996e-06 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670945+0000", 2024-09-16T14:08:09.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8599999999999998e-06 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670949+0000", 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.653e-06 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721301+0000", 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050352791000000001 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721435+0000", 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013400799999999999 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721572+0000", 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013667799999999999 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102417 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 156672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7440381120000001, 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050881124999999999, 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102417 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670954+0000", 2024-09-16T14:08:09.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670956+0000", 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.215e-06 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670962+0000", 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8320000000000002e-06 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670967+0000", 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5539999999999997e-06 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721604+0000", 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050636674 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721699+0000", 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5441999999999999e-05 2024-09-16T14:08:09.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721834+0000", 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134704 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102420 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 3072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:09.429 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7439665690000004, 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051052476999999999, 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102420 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671025+0000", 2024-09-16T14:08:09.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.018e-06 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671030+0000", 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8189999999999999e-06 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671042+0000", 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1902000000000001e-05 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671048+0000", 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2149999999999998e-06 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721866+0000", 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050817648 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721964+0000", 2024-09-16T14:08:09.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7746999999999998e-05 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.722077+0000", 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011312800000000001 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103091 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7215909490000003, 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050794981000000003, 2024-09-16T14:08:09.434 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103091 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7800000000000002e-07 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693403+0000", 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2340000000000001e-06 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693408+0000", 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.806e-06 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693412+0000", 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5939999999999998e-06 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744002+0000", 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050590334000000001 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744093+0000", 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0394000000000002e-05 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744195+0000", 2024-09-16T14:08:09.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102241 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103097 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7215747810000002, 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050990884, 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103097 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:09.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693839+0000", 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00042325899999999999 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693844+0000", 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9640000000000002e-06 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693855+0000", 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0951999999999999e-05 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693863+0000", 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2309999999999999e-06 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744227+0000", 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050364642000000001 2024-09-16T14:08:09.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744304+0000", 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7088000000000005e-05 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744407+0000", 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102748 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103100 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 204800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7211217809999999, 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050768523000000003, 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103100 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.444 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693870+0000", 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693889+0000", 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9040999999999999e-05 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693902+0000", 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3298e-05 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693907+0000", 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9690000000000002e-06 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744435+0000", 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050527739000000002 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744523+0000", 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8225000000000003e-05 2024-09-16T14:08:09.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744638+0000", 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011479700000000001 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103114 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7210783589999998, 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050930699000000003, 2024-09-16T14:08:09.448 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103114 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-07 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693916+0000", 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4540000000000001e-06 2024-09-16T14:08:09.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693922+0000", 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693927+0000", 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9089999999999999e-06 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744661+0000", 2024-09-16T14:08:09.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050733893000000002 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744735+0000", 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4637999999999999e-05 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744843+0000", 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107952 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.452 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103119 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7210600779999998, 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051196642000000001, 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.453 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103119 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693933+0000", 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0540000000000002e-06 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693939+0000", 2024-09-16T14:08:09.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2090000000000002e-06 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693944+0000", 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744904+0000", 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050960022000000001 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745010+0000", 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106399 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745127+0000", 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117451 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103122 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 168960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7210431489999998, 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051372329000000001, 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103122 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6199999999999998e-07 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693951+0000", 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3029999999999998e-06 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693956+0000", 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6829999999999996e-06 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693960+0000", 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6660000000000001e-06 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745168+0000", 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051207616999999997 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745244+0000", 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6412e-05 2024-09-16T14:08:09.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745320+0000", 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6186000000000005e-05 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103127 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.462 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7210271309999996, 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051736246, 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103127 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:09.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693966+0000", 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2120000000000002e-06 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693972+0000", 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8680000000000004e-06 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693976+0000", 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4850000000000001e-06 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745485+0000", 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051508784000000002 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745581+0000", 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6396000000000003e-05 2024-09-16T14:08:09.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745700+0000", 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011909 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103135 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 20480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7210114599999997, 2024-09-16T14:08:09.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051887555000000002, 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103135 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693980+0000", 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693982+0000", 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8300000000000001e-06 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693989+0000", 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.691e-06 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693992+0000", 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3699999999999999e-06 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745733+0000", 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051741405999999997 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745786+0000", 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2967000000000003e-05 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745867+0000", 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0668999999999996e-05 2024-09-16T14:08:09.471 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103142 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7209952309999998, 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052104512999999998, 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.472 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103142 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697955+0000", 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039591230000000002 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697960+0000", 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000001e-06 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697971+0000", 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1094000000000001e-05 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697977+0000", 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7989999999999999e-06 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745893+0000", 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047915755999999997 2024-09-16T14:08:09.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745972+0000", 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9406999999999994e-05 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.746100+0000", 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000127961 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103237 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 12288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7166723089999998, 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050774314000000001, 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103237 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701919+0000", 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036008490000000002 2024-09-16T14:08:09.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701930+0000", 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0882e-05 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701940+0000", 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9620000000000005e-06 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701953+0000", 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945999999999999e-05 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.748858+0000", 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046904281999999999 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.748955+0000", 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7583000000000004e-05 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.749093+0000", 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013781000000000001 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103351 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 184320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7126584520000003, 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.05063119, 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.481 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103351 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702333+0000", 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.72e-07 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702335+0000", 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8220000000000001e-06 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702340+0000", 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0579999999999997e-06 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702344+0000", 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2479999999999998e-06 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752784+0000", 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050439954000000002 2024-09-16T14:08:09.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752882+0000", 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752964+0000", 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1549000000000001e-05 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103357 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.712642958, 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050776609, 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103357 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705848+0000", 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035001839999999999 2024-09-16T14:08:09.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705854+0000", 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4779999999999999e-06 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705863+0000", 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5529999999999998e-06 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705869+0000", 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1800000000000001e-06 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752985+0000", 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04711605 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.753035+0000", 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9997000000000003e-05 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.753125+0000", 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9166999999999995e-05 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104426 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 152576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6786967959999997, 2024-09-16T14:08:09.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050895079000000003, 2024-09-16T14:08:09.491 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.491 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.491 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104426 2024-09-16T14:08:09.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.491 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5999999999999999e-07 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736296+0000", 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9539999999999998e-06 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736302+0000", 2024-09-16T14:08:09.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1599999999999997e-06 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736305+0000", 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1609999999999999e-06 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787004+0000", 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050698852000000003 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787087+0000", 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3418999999999995e-05 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787189+0000", 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102173 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104434 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 229376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.678681901, 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051134456000000002, 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104434 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:09.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736311+0000", 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736317+0000", 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3709999999999999e-06 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736320+0000", 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4230000000000001e-06 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787233+0000", 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050912675999999997 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787329+0000", 2024-09-16T14:08:09.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6628999999999999e-05 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787443+0000", 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011399400000000001 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104446 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 4096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:09.500 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6783414370000003, 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050967213999999997, 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104446 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:09.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736650+0000", 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6700000000000002e-07 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736655+0000", 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6940000000000003e-06 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736665+0000", 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0121e-05 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736671+0000", 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4609999999999999e-06 2024-09-16T14:08:09.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787463+0000", 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050792342999999997 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787528+0000", 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4453000000000001e-05 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787617+0000", 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9174999999999999e-05 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104454 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6783048130000005, 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051215203000000001, 2024-09-16T14:08:09.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104454 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736687+0000", 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0599999999999999e-07 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736690+0000", 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:09.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736697+0000", 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736701+0000", 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5419999999999999e-06 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787711+0000", 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051009546000000003 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787799+0000", 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7961999999999996e-05 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787901+0000", 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102416 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104465 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 193536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.6773042079999998, 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050750456999999999, 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104465 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:09.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6300000000000005e-07 2024-09-16T14:08:09.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737690+0000", 2024-09-16T14:08:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7310000000000002e-06 2024-09-16T14:08:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737696+0000", 2024-09-16T14:08:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1530000000000002e-06 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737700+0000", 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8909999999999997e-06 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788242+0000", 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050542301999999997 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788329+0000", 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6527999999999996e-05 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788437+0000", 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108189 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.515 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:09.517 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:09.517 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:09.510+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:09.606 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:09.606 INFO:teuthology.orchestra.run.smithi079.stdout:2749 punch from 0x1318ee5 to 0x1323077, (0xa192 bytes) 2024-09-16T14:08:09.618 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:09.610+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:09.639 INFO:teuthology.orchestra.run.smithi079.stdout:2750 write 0xe8ed129 thru 0xe8f972f (0xc607 bytes) 2024-09-16T14:08:09.642 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:09.646 INFO:teuthology.orchestra.run.smithi079.stdout:2751 writesame 0x6d04ca2 thru 0x6d04d71 data_size 0x8(0xd0 bytes) 2024-09-16T14:08:09.653 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:09.665 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:09.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:09.665 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:09.665 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:09.670 INFO:teuthology.orchestra.run.smithi079.stdout:2754 trunc from 0xe8f9730 to 0x64541ab 2024-09-16T14:08:09.674 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:09.709 INFO:teuthology.orchestra.run.smithi079.stdout:2755 writesame 0x74fb00e thru 0x74fc890 data_size 0x5(0x1883 bytes) 2024-09-16T14:08:09.709 INFO:teuthology.orchestra.run.smithi079.stdout:2756 write 0x711d8aa thru 0x71212d5 (0x3a2c bytes) 2024-09-16T14:08:09.719 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:09.717+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:09.741 INFO:teuthology.orchestra.run.smithi079.stdout:2757 write 0xde927a0 thru 0xde9e351 (0xbbb2 bytes) 2024-09-16T14:08:09.746 INFO:teuthology.orchestra.run.smithi079.stdout:2758 read 0xab187e5 thru 0xab239d5 (0xb1f1 bytes) 2024-09-16T14:08:09.756 INFO:teuthology.orchestra.run.smithi079.stdout:2759 trunc from 0xde9e352 to 0xaed7587 2024-09-16T14:08:09.775 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:09.775 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:09.777 INFO:teuthology.orchestra.run.smithi079.stdout:2762 trunc from 0xaed7587 to 0xdc07509 2024-09-16T14:08:09.781 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:09.785 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:09.787 INFO:teuthology.orchestra.run.smithi079.stdout:2764 write 0x6b3cb37 thru 0x6b46f86 (0xa450 bytes) 2024-09-16T14:08:09.795 INFO:teuthology.orchestra.run.smithi079.stdout:2765 writesame 0x6b40e3e thru 0x6b4743a data_size 0x9(0x65fd bytes) 2024-09-16T14:08:09.799 INFO:teuthology.orchestra.run.smithi079.stdout:2766 punch from 0x93b0484 to 0x93bb8a9, (0xb425 bytes) 2024-09-16T14:08:09.803 INFO:teuthology.orchestra.run.smithi079.stdout:2767 trunc from 0xdc07509 to 0xbbc0d82 2024-09-16T14:08:09.803 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:09.813 INFO:teuthology.orchestra.run.smithi079.stdout:2768 read 0xa244d25 thru 0xa24b3cf (0x66ab bytes) 2024-09-16T14:08:09.814 INFO:teuthology.orchestra.run.smithi079.stdout:2769 write 0x2b7a0e7 thru 0x2b86ec5 (0xcddf bytes) 2024-09-16T14:08:09.817 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:09.817 INFO:teuthology.orchestra.run.smithi079.stdout:2772 write 0x6b7c907 thru 0x6b8b9d2 (0xf0cc bytes) 2024-09-16T14:08:09.820 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:09.814+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:09.823 INFO:teuthology.orchestra.run.smithi079.stdout:2773 read 0xba06dfe thru 0xba0b769 (0x496c bytes) 2024-09-16T14:08:09.824 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:09.824 INFO:teuthology.orchestra.run.smithi079.stdout:2774 read 0x46b1851 thru 0x46b6753 (0x4f03 bytes) 2024-09-16T14:08:09.824 INFO:teuthology.orchestra.run.smithi079.stdout:2776 punch from 0x98e9e3e to 0x98f8f90, (0xf152 bytes) 2024-09-16T14:08:09.824 INFO:teuthology.orchestra.run.smithi079.stdout:2777 read 0xfd0534 thru 0xfdd340 (0xce0d bytes) 2024-09-16T14:08:09.826 INFO:teuthology.orchestra.run.smithi079.stdout:2778 read 0x19b503a thru 0x19b9360 (0x4327 bytes) 2024-09-16T14:08:09.835 INFO:teuthology.orchestra.run.smithi079.stdout:2779 write 0x7fd19d6 thru 0x7fd43cf (0x29fa bytes) 2024-09-16T14:08:09.852 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:09.856 INFO:teuthology.orchestra.run.smithi079.stdout:2780 writesame 0xe51b68e thru 0xe527a6d data_size 0x10(0xc3e0 bytes) 2024-09-16T14:08:09.856 INFO:teuthology.orchestra.run.smithi079.stdout:2781 read 0x5db6608 thru 0x5dc2fb4 (0xc9ad bytes) 2024-09-16T14:08:09.857 INFO:teuthology.orchestra.run.smithi079.stdout:2784 writesame 0x14cd401 thru 0x14d697b data_size 0x1(0x957b bytes) 2024-09-16T14:08:09.874 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:09.890 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:09.891 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.512663514, 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:08:09.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.5087320960000001, 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:08:09.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:09.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:08:09.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:08:09.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.508707904, 2024-09-16T14:08:09.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:08:09.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:08:09.906 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.504988633, 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:08:09.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:09.911 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.5010718679999999, 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:09.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:08:09.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.496689589, 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.916 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:09.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:08:09.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4966657480000001, 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.921 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.925 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.49665056, 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:09.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:08:09.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4966359739999999, 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:08:09.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.934 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4965323239999999, 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:08:09.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.496504598, 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:09.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:08:09.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.493830982, 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.944 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:08:09.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4938042679999999, 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.948 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:08:09.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.493764101, 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4936743859999999, 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:08:09.957 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:08:09.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.493265582, 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:08:09.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:08:09.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4931653979999999, 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:09.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.970 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4922667600000001, 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:08:09.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.4850680279999999, 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:08:09.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.482899736, 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:08:09.980 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:09.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:09.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:09.985 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:09.986 INFO:teuthology.orchestra.run.smithi079.stdout:2785 clone 25 order 25 su 2097152 sc 3 2024-09-16T14:08:09.987 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:09.987 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:09.987 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:09.988 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:09.989 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:09.982+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:09.989 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:09.989 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:09.989 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:09.989 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:09.989 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:09.990 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:09.990 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:10.064 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:10.158 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:10.159 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:10.150+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:10.258 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:10.250+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:10.269 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:10.269 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:10.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:10.357 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:10.350+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:10.374 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone24 from 0xe527a6e (overlap 0x2989084) to 0x84e9b4 2024-09-16T14:08:10.381 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:10.381 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:10.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:10.458 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:10.450+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:10.522 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:10.523 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:10.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:10.559 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:10.554+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:10.634 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:10.635 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:10.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:10.660 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:10.657+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:10.761 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:10.761 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:10.761 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:10.754+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:10.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:10.862 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:10.854+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:10.873 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:10.873 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:10.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:10.962 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:10.961+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:10.972 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-16T14:08:10.996 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:10.996 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:11.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:11.050 INFO:teuthology.orchestra.run.smithi079.stdout:2786 write 0x7bc3908 thru 0x7bca222 (0x691b bytes) 2024-09-16T14:08:11.063 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:11.058+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:11.076 INFO:teuthology.orchestra.run.smithi079.stdout:2787 trunc from 0xe527a6e to 0xb30560e 2024-09-16T14:08:11.098 INFO:teuthology.orchestra.run.smithi079.stdout:2788 write 0x2952896 thru 0x295af09 (0x8674 bytes) 2024-09-16T14:08:11.151 INFO:teuthology.orchestra.run.smithi079.stdout:2789 write 0x72e0f96 thru 0x72ec2a5 (0xb310 bytes) 2024-09-16T14:08:11.155 INFO:teuthology.orchestra.run.smithi079.stdout:2790 writesame 0x675730f thru 0x676668a data_size 0x2(0xf37c bytes) 2024-09-16T14:08:11.164 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:11.158+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:11.165 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:11.165 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:11.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:11.178 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:11.178 INFO:teuthology.orchestra.run.smithi079.stdout:2793 write 0x2b6dbd3 thru 0x2b7bd16 (0xe144 bytes) 2024-09-16T14:08:11.238 INFO:teuthology.orchestra.run.smithi079.stdout:2794 read 0x5c98ed thru 0x5d1f9a (0x86ae bytes) 2024-09-16T14:08:11.239 INFO:teuthology.orchestra.run.smithi079.stdout:2795 read 0x99205b thru 0x993850 (0x17f6 bytes) 2024-09-16T14:08:11.242 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:11.242 INFO:teuthology.orchestra.run.smithi079.stdout:2799 write 0x92e2755 thru 0x92e4b9d (0x2449 bytes) 2024-09-16T14:08:11.265 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:11.258+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:11.267 INFO:teuthology.orchestra.run.smithi079.stdout:2800 trunc from 0xb30560e to 0x9ead7cd 2024-09-16T14:08:11.302 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:11.302 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:11.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:11.335 INFO:teuthology.orchestra.run.smithi079.stdout:2801 write 0x4ede5f5 thru 0x4eed844 (0xf250 bytes) 2024-09-16T14:08:11.338 INFO:teuthology.orchestra.run.smithi079.stdout:2802 punch from 0x5a092d4 to 0x5a0ea3e, (0x576a bytes) 2024-09-16T14:08:11.338 INFO:teuthology.orchestra.run.smithi079.stdout:2803 read 0x1221639 thru 0x122791d (0x62e5 bytes) 2024-09-16T14:08:11.339 INFO:teuthology.orchestra.run.smithi079.stdout:2804 read 0x3099ee1 thru 0x309a87d (0x99d bytes) 2024-09-16T14:08:11.340 INFO:teuthology.orchestra.run.smithi079.stdout:2805 write 0xcffe65 thru 0xd06bdc (0x6d78 bytes) 2024-09-16T14:08:11.343 INFO:teuthology.orchestra.run.smithi079.stdout:2810 writesame 0x92a2506 thru 0x92a5c80 data_size 0x1(0x377b bytes) 2024-09-16T14:08:11.355 INFO:teuthology.orchestra.run.smithi079.stdout:2811 trunc from 0x9ead7cd to 0x997333a 2024-09-16T14:08:11.365 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:11.361+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:11.371 INFO:teuthology.orchestra.run.smithi079.stdout:2812 read 0x49814f7 thru 0x4986da3 (0x58ad bytes) 2024-09-16T14:08:11.371 INFO:teuthology.orchestra.run.smithi079.stdout:2813 read 0x34281e6 thru 0x3435b3e (0xd959 bytes) 2024-09-16T14:08:11.373 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:11.373 INFO:teuthology.orchestra.run.smithi079.stdout:2816 punch from 0x3f13b08 to 0x3f192cd, (0x57c5 bytes) 2024-09-16T14:08:11.373 INFO:teuthology.orchestra.run.smithi079.stdout:2817 write 0x7ee37a5 thru 0x7ee5fc5 (0x2821 bytes) 2024-09-16T14:08:11.376 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:11.377 INFO:teuthology.orchestra.run.smithi079.stdout:2819 read 0x7b9986d thru 0x7ba6eea (0xd67e bytes) 2024-09-16T14:08:11.377 INFO:teuthology.orchestra.run.smithi079.stdout:2823 punch from 0x84ec8ea to 0x84f09d4, (0x40ea bytes) 2024-09-16T14:08:11.378 INFO:teuthology.orchestra.run.smithi079.stdout:2824 read 0x3589683 thru 0x358bedd (0x285b bytes) 2024-09-16T14:08:11.378 INFO:teuthology.orchestra.run.smithi079.stdout:2825 punch from 0x2152988 to 0x215dcb4, (0xb32c bytes) 2024-09-16T14:08:11.378 INFO:teuthology.orchestra.run.smithi079.stdout:2826 punch from 0x2cd208d to 0x2cdf70a, (0xd67d bytes) 2024-09-16T14:08:11.393 INFO:teuthology.orchestra.run.smithi079.stdout:2827 write 0xb6d7a1a thru 0xb6e3f3d (0xc524 bytes) 2024-09-16T14:08:11.397 INFO:teuthology.orchestra.run.smithi079.stdout:2828 clone 26 order 25 su 1048576 sc 14 2024-09-16T14:08:11.407 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:11.407 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:11.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:11.466 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:11.458+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:11.559 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:11.559 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:11.567 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:11.562+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:11.568 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:11.669 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:11.662+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:11.677 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:11.677 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:11.687 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:11.770 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:11.762+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:11.789 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:11.789 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:11.801 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:11.871 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:11.869+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:11.944 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:11.944 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:11.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:11.972 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:11.969+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:11.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:12.049 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:12.049 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:12.057 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:12.072 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:12.069+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:12.137 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:12.157 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:12.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:08:12.163 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:12.163 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:12.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:12.174 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:12.169+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:12.177 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:12.198 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:12.226 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:12.253 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:12.273 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:12.269+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102411 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6007144540000002, 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050639235999999997, 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.277 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102411 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:12.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0499999999999999e-07 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670939+0000", 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8409999999999996e-06 2024-09-16T14:08:12.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670945+0000", 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8599999999999998e-06 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670949+0000", 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.653e-06 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721301+0000", 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050352791000000001 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721435+0000", 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013400799999999999 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721572+0000", 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013667799999999999 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.281 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102417 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 156672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6006945619999993, 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050881124999999999, 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102417 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670954+0000", 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670956+0000", 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.215e-06 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670962+0000", 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8320000000000002e-06 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670967+0000", 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5539999999999997e-06 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721604+0000", 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050636674 2024-09-16T14:08:12.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721699+0000", 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5441999999999999e-05 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721834+0000", 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134704 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102420 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 3072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.6006230190000004, 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051052476999999999, 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102420 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.287 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671025+0000", 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.018e-06 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671030+0000", 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8189999999999999e-06 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671042+0000", 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1902000000000001e-05 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671048+0000", 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2149999999999998e-06 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721866+0000", 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050817648 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721964+0000", 2024-09-16T14:08:12.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7746999999999998e-05 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.722077+0000", 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011312800000000001 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103091 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5782473990000003, 2024-09-16T14:08:12.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050794981000000003, 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103091 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7800000000000002e-07 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693403+0000", 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2340000000000001e-06 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693408+0000", 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.806e-06 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693412+0000", 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5939999999999998e-06 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744002+0000", 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050590334000000001 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744093+0000", 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0394000000000002e-05 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744195+0000", 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102241 2024-09-16T14:08:12.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103097 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5782312310000002, 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050990884, 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103097 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:12.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693839+0000", 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00042325899999999999 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693844+0000", 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9640000000000002e-06 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693855+0000", 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0951999999999999e-05 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693863+0000", 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2309999999999999e-06 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744227+0000", 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050364642000000001 2024-09-16T14:08:12.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744304+0000", 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7088000000000005e-05 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744407+0000", 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102748 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103100 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 204800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5777782309999999, 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050768523000000003, 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103100 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693870+0000", 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693889+0000", 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9040999999999999e-05 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693902+0000", 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3298e-05 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693907+0000", 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9690000000000002e-06 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744435+0000", 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050527739000000002 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744523+0000", 2024-09-16T14:08:12.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8225000000000003e-05 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744638+0000", 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011479700000000001 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103114 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.305 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5777348090000007, 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050930699000000003, 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103114 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:12.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-07 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693916+0000", 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4540000000000001e-06 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693922+0000", 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693927+0000", 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9089999999999999e-06 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744661+0000", 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050733893000000002 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744735+0000", 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4637999999999999e-05 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744843+0000", 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107952 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103119 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5777165279999998, 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051196642000000001, 2024-09-16T14:08:12.310 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103119 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693933+0000", 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0540000000000002e-06 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693939+0000", 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2090000000000002e-06 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693944+0000", 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744904+0000", 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050960022000000001 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745010+0000", 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106399 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745127+0000", 2024-09-16T14:08:12.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117451 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103122 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 168960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5776995990000007, 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051372329000000001, 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103122 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6199999999999998e-07 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693951+0000", 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3029999999999998e-06 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693956+0000", 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6829999999999996e-06 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693960+0000", 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6660000000000001e-06 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745168+0000", 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051207616999999997 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745244+0000", 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6412e-05 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745320+0000", 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6186000000000005e-05 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103127 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5776835810000005, 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051736246, 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103127 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693966+0000", 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2120000000000002e-06 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693972+0000", 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8680000000000004e-06 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693976+0000", 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4850000000000001e-06 2024-09-16T14:08:12.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745485+0000", 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051508784000000002 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745581+0000", 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6396000000000003e-05 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745700+0000", 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011909 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103135 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 20480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5776679100000006, 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051887555000000002, 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103135 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:12.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693980+0000", 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693982+0000", 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8300000000000001e-06 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693989+0000", 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.691e-06 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693992+0000", 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3699999999999999e-06 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745733+0000", 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051741405999999997 2024-09-16T14:08:12.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745786+0000", 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2967000000000003e-05 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745867+0000", 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0668999999999996e-05 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103142 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5776516810000007, 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052104512999999998, 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103142 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697955+0000", 2024-09-16T14:08:12.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039591230000000002 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697960+0000", 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000001e-06 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697971+0000", 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1094000000000001e-05 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697977+0000", 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7989999999999999e-06 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745893+0000", 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047915755999999997 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745972+0000", 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9406999999999994e-05 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.746100+0000", 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000127961 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103237 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 12288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5733287590000007, 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050774314000000001, 2024-09-16T14:08:12.333 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103237 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701919+0000", 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036008490000000002 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701930+0000", 2024-09-16T14:08:12.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0882e-05 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701940+0000", 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9620000000000005e-06 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701953+0000", 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945999999999999e-05 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.748858+0000", 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046904281999999999 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.748955+0000", 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7583000000000004e-05 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.749093+0000", 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013781000000000001 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103351 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 184320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5693149020000003, 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.05063119, 2024-09-16T14:08:12.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103351 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702333+0000", 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.72e-07 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702335+0000", 2024-09-16T14:08:12.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8220000000000001e-06 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702340+0000", 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0579999999999997e-06 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702344+0000", 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2479999999999998e-06 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752784+0000", 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050439954000000002 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752882+0000", 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752964+0000", 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1549000000000001e-05 2024-09-16T14:08:12.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103357 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5692994079999991, 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050776609, 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103357 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:12.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705848+0000", 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035001839999999999 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705854+0000", 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4779999999999999e-06 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705863+0000", 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5529999999999998e-06 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705869+0000", 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1800000000000001e-06 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752985+0000", 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04711605 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.753035+0000", 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9997000000000003e-05 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.753125+0000", 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9166999999999995e-05 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104426 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 152576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5353532459999997, 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050895079000000003, 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104426 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:12.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5999999999999999e-07 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736296+0000", 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9539999999999998e-06 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736302+0000", 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1599999999999997e-06 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736305+0000", 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1609999999999999e-06 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787004+0000", 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050698852000000003 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787087+0000", 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3418999999999995e-05 2024-09-16T14:08:12.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787189+0000", 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102173 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104434 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 229376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.535338351, 2024-09-16T14:08:12.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051134456000000002, 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104434 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:12.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736311+0000", 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736317+0000", 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3709999999999999e-06 2024-09-16T14:08:12.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736320+0000", 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4230000000000001e-06 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787233+0000", 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050912675999999997 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787329+0000", 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6628999999999999e-05 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787443+0000", 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011399400000000001 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104446 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 4096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.357 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5349978869999994, 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050967213999999997, 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104446 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736650+0000", 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6700000000000002e-07 2024-09-16T14:08:12.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736655+0000", 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6940000000000003e-06 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736665+0000", 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0121e-05 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736671+0000", 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4609999999999999e-06 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787463+0000", 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050792342999999997 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787528+0000", 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4453000000000001e-05 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787617+0000", 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9174999999999999e-05 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104454 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5349612629999996, 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051215203000000001, 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104454 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736687+0000", 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0599999999999999e-07 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736690+0000", 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736697+0000", 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736701+0000", 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5419999999999999e-06 2024-09-16T14:08:12.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787711+0000", 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051009546000000003 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787799+0000", 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7961999999999996e-05 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787901+0000", 2024-09-16T14:08:12.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102416 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104465 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 193536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5339606579999998, 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050750456999999999, 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104465 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:12.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6300000000000005e-07 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737690+0000", 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7310000000000002e-06 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737696+0000", 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1530000000000002e-06 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737700+0000", 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8909999999999997e-06 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788242+0000", 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050542301999999997 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788329+0000", 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6527999999999996e-05 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788437+0000", 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108189 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.371 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:12.372 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:12.373 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:12.373 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:12.374 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:12.374 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:12.374 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:12.374 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:12.374 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:12.375 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:12.375 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:12.375 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:12.375 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:12.375 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone25 from 0xb6e3f3e (overlap 0x997333a) to 0x0 2024-09-16T14:08:12.376 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:12.370+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:12.475 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:12.475 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:12.475 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:12.476 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:12.477 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:12.470+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:12.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:12.577 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:12.570+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:12.588 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:12.589 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:12.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:12.678 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:12.674+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:12.752 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:12.752 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:12.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:12.752 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:12.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3721016480000001, 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:08:12.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:08:12.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:12.757 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3681702299999996, 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:12.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:08:12.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3681460379999999, 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.762 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:12.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:08:12.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3644267670000003, 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:08:12.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3605100019999998, 2024-09-16T14:08:12.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:08:12.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3561277230000002, 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:12.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:08:12.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3561038820000002, 2024-09-16T14:08:12.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:08:12.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3560886940000003, 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:08:12.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.790 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3560741079999996, 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:08:12.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3559704579999998, 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.795 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:08:12.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3559427319999999, 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.800 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3532691159999999, 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.804 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:08:12.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3532424020000002, 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:08:12.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3532022350000004, 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.817 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3531125199999998, 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:08:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3527037159999997, 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:12.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:08:12.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3526035319999998, 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:08:12.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:08:12.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.351704894, 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:08:12.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3445061620000001, 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:12.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:08:12.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3423378699999997, 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:08:12.843 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:12.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:08:12.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:12.848 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:12.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:12.850 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:12.842+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:12.878 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-16T14:08:12.928 INFO:teuthology.orchestra.run.smithi079.stdout:2829 read 0x1069d23 thru 0x1079382 (0xf660 bytes) 2024-09-16T14:08:12.942 INFO:teuthology.orchestra.run.smithi079.stdout:2830 read 0x6e5c1ae thru 0x6e67d28 (0xbb7b bytes) 2024-09-16T14:08:12.949 INFO:teuthology.orchestra.run.smithi079.stdout:2831 trunc from 0xb6e3f3e to 0x9ecc37a 2024-09-16T14:08:12.950 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:12.942+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:13.005 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:13.006 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:13.006 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:13.006 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:13.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:13.029 INFO:teuthology.orchestra.run.smithi079.stdout:2833 read 0xdf57a6 thru 0xe04426 (0xec81 bytes) 2024-09-16T14:08:13.039 INFO:teuthology.orchestra.run.smithi079.stdout:2838 writesame 0xa550867 thru 0xa552ca7 data_size 0x1(0x2441 bytes) 2024-09-16T14:08:13.051 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:13.046+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:13.054 INFO:teuthology.orchestra.run.smithi079.stdout:2839 write 0xc2d4281 thru 0xc2d9ad9 (0x5859 bytes) 2024-09-16T14:08:13.057 INFO:teuthology.orchestra.run.smithi079.stdout:2840 write 0x63928ee thru 0x6394844 (0x1f57 bytes) 2024-09-16T14:08:13.060 INFO:teuthology.orchestra.run.smithi079.stdout:2841 write 0x95fe037 thru 0x96082a9 (0xa273 bytes) 2024-09-16T14:08:13.067 INFO:teuthology.orchestra.run.smithi079.stdout:2843 trunc from 0xc2d9ada to 0xa92852c 2024-09-16T14:08:13.087 INFO:teuthology.orchestra.run.smithi079.stdout:2844 write 0x5b56197 thru 0x5b63601 (0xd46b bytes) 2024-09-16T14:08:13.090 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:08:13.090 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.090 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.107 INFO:teuthology.orchestra.run.smithi079.stdout:2849 write 0xc1a61ad thru 0xc1a783c (0x1690 bytes) 2024-09-16T14:08:13.111 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.111 INFO:teuthology.orchestra.run.smithi079.stdout:2852 writesame 0xac165f9 thru 0xac1c5fe data_size 0x3(0x6006 bytes) 2024-09-16T14:08:13.116 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:13.116 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:13.117 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:13.117 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:13.117 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:13.117 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.126 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:13.134 INFO:teuthology.orchestra.run.smithi079.stdout:2855 write 0xe7907bd thru 0xe798197 (0x79db bytes) 2024-09-16T14:08:13.137 INFO:teuthology.orchestra.run.smithi079.stdout:2856 punch from 0x8cf3c7 to 0x8deda6, (0xf9df bytes) 2024-09-16T14:08:13.137 INFO:teuthology.orchestra.run.smithi079.stdout:2857 read 0x67663 thru 0x6adef (0x378d bytes) 2024-09-16T14:08:13.152 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:13.149+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:13.162 INFO:teuthology.orchestra.run.smithi079.stdout:2858 trunc from 0xe798198 to 0xaac600 2024-09-16T14:08:13.189 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.189 INFO:teuthology.orchestra.run.smithi079.stdout:2861 punch from 0x2484a2 to 0x252a1c, (0xa57a bytes) 2024-09-16T14:08:13.189 INFO:teuthology.orchestra.run.smithi079.stdout:2863 punch from 0x4db90b to 0x4e439c, (0x8a91 bytes) 2024-09-16T14:08:13.190 INFO:teuthology.orchestra.run.smithi079.stdout:2864 punch from 0x331d48 to 0x33e8c8, (0xcb80 bytes) 2024-09-16T14:08:13.190 INFO:teuthology.orchestra.run.smithi079.stdout:2865 read 0x6e8209 thru 0x6f380b (0xb603 bytes) 2024-09-16T14:08:13.190 INFO:teuthology.orchestra.run.smithi079.stdout:2867 read 0x7d0299 thru 0x7ddbae (0xd916 bytes) 2024-09-16T14:08:13.191 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.206 INFO:teuthology.orchestra.run.smithi079.stdout:2869 write 0x396a9bd thru 0x397a224 (0xf868 bytes) 2024-09-16T14:08:13.225 INFO:teuthology.orchestra.run.smithi079.stdout:2870 write 0x5e2bea5 thru 0x5e2c06f (0x1cb bytes) 2024-09-16T14:08:13.249 INFO:teuthology.orchestra.run.smithi079.stdout:2872 write 0xa8d0273 thru 0xa8da075 (0x9e03 bytes) 2024-09-16T14:08:13.253 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:13.249+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:13.259 INFO:teuthology.orchestra.run.smithi079.stdout:2873 trunc from 0xa8da076 to 0xda672f3 2024-09-16T14:08:13.264 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:13.264 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:13.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:13.264 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:13.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:08:13.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.848973405000002, 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:08:13.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:08:13.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.011011082, 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:08:13.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:08:13.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:89451 3.b 3:d6399513:::rbd_data.1136f7114f14.000000000000007b:head [delete] snapc 18=[18] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:00.254760+0000", 2024-09-16T14:08:13.277 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.004149330000001, 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013757736, 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 89451 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254760+0000", 2024-09-16T14:08:13.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254760+0000", 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254761+0000", 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254764+0000", 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.413e-06 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254768+0000", 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7500000000000003e-06 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.254772+0000", 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2629999999999999e-06 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.263916+0000", 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091445490000000001 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.263987+0000", 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0727000000000002e-05 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.264204+0000", 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000217147 2024-09-16T14:08:13.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.267361+0000", 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031571910000000002 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.268404+0000", 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001042591 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.268468+0000", 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4455000000000005e-05 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:00.268518+0000", 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9945999999999999e-05 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123407 3.b 3:d20e38df:::rbd_object_map.11368cf3b67c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.283 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.356802+0000", 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.9021077349999995, 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015363415, 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123407 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.356802+0000", 2024-09-16T14:08:13.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.356802+0000", 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.356802+0000", 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8100000000000003e-07 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.356806+0000", 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7179999999999998e-06 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.356814+0000", 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5889999999999997e-06 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.356818+0000", 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5549999999999996e-06 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.365328+0000", 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085098900000000009 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.365454+0000", 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125907 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.366070+0000", 2024-09-16T14:08:13.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00061548799999999997 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372075+0000", 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060051310000000004 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372115+0000", 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9845000000000002e-05 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372145+0000", 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0403999999999999e-05 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372165+0000", 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0407e-05 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.289 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123660 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.358849+0000", 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.9000608269999999, 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013229108, 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123660 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358849+0000", 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358849+0000", 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358850+0000", 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6600000000000005e-07 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358854+0000", 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358878+0000", 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4397999999999999e-05 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358885+0000", 2024-09-16T14:08:13.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1790000000000002e-06 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.371911+0000", 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01302561 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.371982+0000", 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1645999999999998e-05 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372078+0000", 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5619999999999996e-05 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123662 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 1638400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.358892+0000", 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.9000176660000001, 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013425495000000001, 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123662 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358892+0000", 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358892+0000", 2024-09-16T14:08:13.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358893+0000", 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358897+0000", 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4580000000000003e-06 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358906+0000", 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358910+0000", 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7849999999999999e-06 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372220+0000", 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013309184999999999 2024-09-16T14:08:13.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372273+0000", 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3133000000000001e-05 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372318+0000", 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4792000000000001e-05 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123664 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 1884160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.358920+0000", 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.89998954, 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013607765000000001, 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123664 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358920+0000", 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358920+0000", 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358921+0000", 2024-09-16T14:08:13.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1800000000000005e-07 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358929+0000", 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8650000000000001e-06 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358937+0000", 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.32e-06 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.358942+0000", 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8790000000000002e-06 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372359+0000", 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013417223000000001 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372458+0000", 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9159000000000004e-05 2024-09-16T14:08:13.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372528+0000", 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9601e-05 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123756 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 278528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.359806+0000", 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8991033509999999, 2024-09-16T14:08:13.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013172791999999999, 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123756 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359806+0000", 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359806+0000", 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359808+0000", 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.265e-06 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359812+0000", 2024-09-16T14:08:13.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6739999999999996e-06 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359838+0000", 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5374000000000001e-05 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359845+0000", 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.413e-06 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372772+0000", 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012926434000000001 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372873+0000", 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101047 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.372979+0000", 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106585 2024-09-16T14:08:13.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123758 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.359852+0000", 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8990577990000004, 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013356272000000001, 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123758 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359852+0000", 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359852+0000", 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359853+0000", 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0299999999999999e-07 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359855+0000", 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8650000000000001e-06 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359861+0000", 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5079999999999996e-06 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359865+0000", 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8369999999999999e-06 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373015+0000", 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013150520000000001 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373114+0000", 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9130999999999997e-05 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373208+0000", 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3807999999999994e-05 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123760 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 983040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.359870+0000", 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8990401690000001, 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013495572000000001, 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123760 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359870+0000", 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359870+0000", 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359870+0000", 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T14:08:13.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359873+0000", 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4760000000000001e-06 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359897+0000", 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4269000000000001e-05 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359906+0000", 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6580000000000008e-06 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373226+0000", 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0133204 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373272+0000", 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6236999999999998e-05 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373365+0000", 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2800999999999995e-05 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123761 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 1196032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.359913+0000", 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8989972279999998, 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013687263, 2024-09-16T14:08:13.317 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123761 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359913+0000", 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359913+0000", 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359913+0000", 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7000000000000004e-07 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359916+0000", 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4789999999999999e-06 2024-09-16T14:08:13.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359923+0000", 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1749999999999999e-06 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.359942+0000", 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9457999999999999e-05 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373401+0000", 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013458296999999999 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373490+0000", 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9631999999999998e-05 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373600+0000", 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109552 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123763 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 1441792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.360012+0000", 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8988981760000003, 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013778680999999999, 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123763 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360012+0000", 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360012+0000", 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360013+0000", 2024-09-16T14:08:13.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.78e-07 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360018+0000", 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3850000000000001e-06 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360030+0000", 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1623999999999999e-05 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360037+0000", 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4109999999999999e-06 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373649+0000", 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013612188000000001 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373718+0000", 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8398999999999993e-05 2024-09-16T14:08:13.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373790+0000", 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2695999999999997e-05 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123764 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 1654784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.360043+0000", 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8988666849999998, 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013940981999999999, 2024-09-16T14:08:13.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123764 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360043+0000", 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360043+0000", 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360044+0000", 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1699999999999998e-07 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360047+0000", 2024-09-16T14:08:13.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4369999999999999e-06 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360055+0000", 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6059999999999997e-06 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360060+0000", 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4e-06 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373839+0000", 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013779099 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373897+0000", 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7988000000000001e-05 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.373984+0000", 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6835000000000005e-05 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123766 3.b 3:d0563915:::rbd_data.113618491ebc.000000000000000d:1a [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.360067+0000", 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8988431449999998, 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014043622, 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123766 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360067+0000", 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360067+0000", 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360067+0000", 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360070+0000", 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6730000000000001e-06 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360078+0000", 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8350000000000004e-06 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360083+0000", 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8060000000000002e-06 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374015+0000", 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013932094000000001 2024-09-16T14:08:13.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374060+0000", 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5627000000000002e-05 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374110+0000", 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9855999999999998e-05 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123772 3.b 3:d20e38df:::rbd_object_map.11368cf3b67c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.360104+0000", 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8988057019999998, 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018187977000000001, 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123772 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360104+0000", 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360104+0000", 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360105+0000", 2024-09-16T14:08:13.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.71e-07 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360110+0000", 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.763e-06 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360120+0000", 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0210000000000001e-05 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360124+0000", 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5990000000000004e-06 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374139+0000", 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014014097999999999 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374214+0000", 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4989999999999999e-05 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374512+0000", 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029880400000000001 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378178+0000", 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003665869 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378218+0000", 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0213000000000003e-05 2024-09-16T14:08:13.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378253+0000", 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4938999999999999e-05 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378292+0000", 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8621000000000002e-05 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123783 3.b 3:d20e38df:::rbd_object_map.11368cf3b67c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.360131+0000", 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8987791529999996, 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018280153, 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123783 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360131+0000", 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360131+0000", 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360131+0000", 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0800000000000004e-07 2024-09-16T14:08:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360135+0000", 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7759999999999999e-06 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360142+0000", 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.481e-06 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360156+0000", 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3426e-05 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374982+0000", 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014825714 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.375037+0000", 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5448999999999997e-05 2024-09-16T14:08:13.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.375333+0000", 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029576599999999999 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378334+0000", 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030013969999999998 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378342+0000", 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.841e-06 2024-09-16T14:08:13.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378371+0000", 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8657999999999999e-05 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378411+0000", 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0037000000000003e-05 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123790 3.b 3:d20e38df:::rbd_object_map.11368cf3b67c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.360162+0000", 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8987477510000002, 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018402908999999999, 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123790 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360162+0000", 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360162+0000", 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360163+0000", 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6499999999999998e-07 2024-09-16T14:08:13.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360166+0000", 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3529999999999999e-06 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360201+0000", 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5222999999999998e-05 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360208+0000", 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5729999999999996e-06 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.375855+0000", 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015647232000000001 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.376049+0000", 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019382899999999999 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.376716+0000", 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066697700000000002 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378423+0000", 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001706709 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378505+0000", 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2793000000000005e-05 2024-09-16T14:08:13.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378533+0000", 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7217999999999999e-05 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378565+0000", 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2237000000000003e-05 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123795 3.b 3:d20e38df:::rbd_object_map.11368cf3b67c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.360216+0000", 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8986938899999997, 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02149802, 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123795 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360216+0000", 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360216+0000", 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360217+0000", 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2130000000000001e-06 2024-09-16T14:08:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360224+0000", 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5050000000000004e-06 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360234+0000", 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0033e-05 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.360239+0000", 2024-09-16T14:08:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1830000000000002e-06 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.377389+0000", 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017150298000000001 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.377449+0000", 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0129e-05 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.377759+0000", 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030963799999999999 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381479+0000", 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037204 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381566+0000", 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6315000000000003e-05 2024-09-16T14:08:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381655+0000", 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9753000000000006e-05 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381714+0000", 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8553e-05 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:130865 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:12.953608+0000", 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.30530158499999999, 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014490482000000001, 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 130865 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.953608+0000", 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.953608+0000", 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.953611+0000", 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3779999999999999e-06 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.958998+0000", 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053876660000000002 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.959052+0000", 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3807999999999997e-05 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.959071+0000", 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9148999999999999e-05 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.959176+0000", 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105345 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.959319+0000", 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014242399999999999 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.959539+0000", 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022002499999999999 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.967755+0000", 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082155660000000005 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.968000+0000", 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024567300000000002 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.968051+0000", 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0319999999999999e-05 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:12.968099+0000", 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8127999999999997e-05 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:131041 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 11534336~2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:13.002518+0000", 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.25639144000000003, 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014152196000000001, 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 131041 2024-09-16T14:08:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.002518+0000", 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.002518+0000", 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.002520+0000", 2024-09-16T14:08:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9549999999999999e-06 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.004976+0000", 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024560480000000002 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.005003+0000", 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6594999999999999e-05 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.005013+0000", 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0023e-05 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.005100+0000", 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6730000000000005e-05 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.005359+0000", 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025981399999999998 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.006056+0000", 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000696241 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.010901+0000", 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048450660000000003 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.016553+0000", 2024-09-16T14:08:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056517529999999998 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.016625+0000", 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2484000000000006e-05 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:13.016670+0000", 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5487e-05 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:13.372 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:13.374 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:13.366+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:13.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:13.375 INFO:teuthology.orchestra.run.smithi079.stdout:2875 trunc from 0xda672f3 to 0xb2eed2c 2024-09-16T14:08:13.375 INFO:teuthology.orchestra.run.smithi079.stdout:2878 read 0x13e2e31 thru 0x13ec590 (0x9760 bytes) 2024-09-16T14:08:13.375 INFO:teuthology.orchestra.run.smithi079.stdout:2879 read 0x1b4ab01 thru 0x1b59e92 (0xf392 bytes) 2024-09-16T14:08:13.375 INFO:teuthology.orchestra.run.smithi079.stdout:2880 read 0x1fe6aa7 thru 0x1fecc04 (0x615e bytes) 2024-09-16T14:08:13.375 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.375 INFO:teuthology.orchestra.run.smithi079.stdout:2884 read 0xa48680a thru 0xa4952da (0xead1 bytes) 2024-09-16T14:08:13.375 INFO:teuthology.orchestra.run.smithi079.stdout:2885 writesame 0xa43697d thru 0xa43e2c6 data_size 0x1e(0x794a bytes) 2024-09-16T14:08:13.375 INFO:teuthology.orchestra.run.smithi079.stdout:2887 writesame 0xaa7d1c4 thru 0xaa8d19b data_size 0xc(0xffd8 bytes) 2024-09-16T14:08:13.376 INFO:teuthology.orchestra.run.smithi079.stdout:2889 punch from 0x98d4b50 to 0x98e2120, (0xd5d0 bytes) 2024-09-16T14:08:13.376 INFO:teuthology.orchestra.run.smithi079.stdout:2890 trunc from 0xb2eed2c to 0xe76dfb8 2024-09-16T14:08:13.376 INFO:teuthology.orchestra.run.smithi079.stdout:2891 trunc from 0xe76dfb8 to 0x1ac93ce 2024-09-16T14:08:13.376 INFO:teuthology.orchestra.run.smithi079.stdout:2892 read 0x1216166 thru 0x121fd4d (0x9be8 bytes) 2024-09-16T14:08:13.390 INFO:teuthology.orchestra.run.smithi079.stdout:2893 trunc from 0x1ac93ce to 0xce6c5d1 2024-09-16T14:08:13.401 INFO:teuthology.orchestra.run.smithi079.stdout:2895 write 0x5be7e44 thru 0x5bf37f1 (0xb9ae bytes) 2024-09-16T14:08:13.403 INFO:teuthology.orchestra.run.smithi079.stdout:2896 read 0x12d10c7 thru 0x12d64d3 (0x540d bytes) 2024-09-16T14:08:13.418 INFO:teuthology.orchestra.run.smithi079.stdout:2897 writesame 0xe7547b9 thru 0xe761379 data_size 0x1(0xcbc1 bytes) 2024-09-16T14:08:13.420 INFO:teuthology.orchestra.run.smithi079.stdout:2898 write 0xc386298 thru 0xc3900ac (0x9e15 bytes) 2024-09-16T14:08:13.427 INFO:teuthology.orchestra.run.smithi079.stdout:2899 write 0xc1c7a5c thru 0xc1ceddf (0x7384 bytes) 2024-09-16T14:08:13.434 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.434 INFO:teuthology.orchestra.run.smithi079.stdout:2903 punch from 0x2ae2323 to 0x2aef926, (0xd603 bytes) 2024-09-16T14:08:13.434 INFO:teuthology.orchestra.run.smithi079.stdout:2904 punch from 0x1b261cc to 0x1b331a3, (0xcfd7 bytes) 2024-09-16T14:08:13.434 INFO:teuthology.orchestra.run.smithi079.stdout:2905 read 0x28be21d thru 0x28c8a9a (0xa87e bytes) 2024-09-16T14:08:13.434 INFO:teuthology.orchestra.run.smithi079.stdout:2906 read 0xd90ed6f thru 0xd91a96a (0xbbfc bytes) 2024-09-16T14:08:13.435 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.435 INFO:teuthology.orchestra.run.smithi079.stdout:2908 read 0x28e8c2e thru 0x28f7b1d (0xeef0 bytes) 2024-09-16T14:08:13.435 INFO:teuthology.orchestra.run.smithi079.stdout:2909 read 0x5f36b7c thru 0x5f3d934 (0x6db9 bytes) 2024-09-16T14:08:13.435 INFO:teuthology.orchestra.run.smithi079.stdout:2910 read 0x3728f13 thru 0x3733151 (0xa23f bytes) 2024-09-16T14:08:13.436 INFO:teuthology.orchestra.run.smithi079.stdout:2911 punch from 0x7f084ea to 0x7f13450, (0xaf66 bytes) 2024-09-16T14:08:13.436 INFO:teuthology.orchestra.run.smithi079.stdout:2913 write 0xdf3d179 thru 0xdf405ea (0x3472 bytes) 2024-09-16T14:08:13.440 INFO:teuthology.orchestra.run.smithi079.stdout:2914 write 0x1f9db1b thru 0x1f9ff7e (0x2464 bytes) 2024-09-16T14:08:13.443 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.443 INFO:teuthology.orchestra.run.smithi079.stdout:2916 punch from 0xa106c29 to 0xa1101d7, (0x95ae bytes) 2024-09-16T14:08:13.443 INFO:teuthology.orchestra.run.smithi079.stdout:2918 read 0x653b730 thru 0x653b9d7 (0x2a8 bytes) 2024-09-16T14:08:13.444 INFO:teuthology.orchestra.run.smithi079.stdout:2919 write 0x641f336 thru 0x642e503 (0xf1ce bytes) 2024-09-16T14:08:13.449 INFO:teuthology.orchestra.run.smithi079.stdout:2920 write 0x87364bb thru 0x873d1c9 (0x6d0f bytes) 2024-09-16T14:08:13.453 INFO:teuthology.orchestra.run.smithi079.stdout:2922 read 0x85fb5ea thru 0x85fcebc (0x18d3 bytes) 2024-09-16T14:08:13.453 INFO:teuthology.orchestra.run.smithi079.stdout:2925 writesame 0x93b986c thru 0x93bb793 data_size 0x4(0x1f28 bytes) 2024-09-16T14:08:13.458 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.458 INFO:teuthology.orchestra.run.smithi079.stdout:2928 punch from 0x4126b33 to 0x412e6fe, (0x7bcb bytes) 2024-09-16T14:08:13.458 INFO:teuthology.orchestra.run.smithi079.stdout:2929 write 0x4cfbed4 thru 0x4d0aa35 (0xeb62 bytes) 2024-09-16T14:08:13.462 INFO:teuthology.orchestra.run.smithi079.stdout:2930 punch from 0x98662de to 0x986c46f, (0x6191 bytes) 2024-09-16T14:08:13.462 INFO:teuthology.orchestra.run.smithi079.stdout:2931 writesame 0x927b29c thru 0x927fb5d data_size 0x43(0x48c2 bytes) 2024-09-16T14:08:13.466 INFO:teuthology.orchestra.run.smithi079.stdout:2932 writesame 0x88d287f thru 0x88da81f data_size 0x1(0x7fa1 bytes) 2024-09-16T14:08:13.474 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:13.473+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:13.480 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:13.481 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:13.481 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:13.481 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:13.486 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.487 INFO:teuthology.orchestra.run.smithi079.stdout:2935 writesame 0x2461011 thru 0x24668ae data_size 0x39(0x589e bytes) 2024-09-16T14:08:13.489 INFO:teuthology.orchestra.run.smithi079.stdout:2936 read 0x72a6a3e thru 0x72b2a35 (0xbff8 bytes) 2024-09-16T14:08:13.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:13.491 INFO:teuthology.orchestra.run.smithi079.stdout:2937 writesame 0x39ceebe thru 0x39d626f data_size 0x3b(0x73b2 bytes) 2024-09-16T14:08:13.494 INFO:teuthology.orchestra.run.smithi079.stdout:2939 read 0x4842ae0 thru 0x4852340 (0xf861 bytes) 2024-09-16T14:08:13.496 INFO:teuthology.orchestra.run.smithi079.stdout:2940 read 0x58891e2 thru 0x588e358 (0x5177 bytes) 2024-09-16T14:08:13.496 INFO:teuthology.orchestra.run.smithi079.stdout:2941 read 0x3844a2 thru 0x385dcd (0x192c bytes) 2024-09-16T14:08:13.498 INFO:teuthology.orchestra.run.smithi079.stdout:2943 trunc from 0xe76137a to 0xde5e6fc 2024-09-16T14:08:13.511 INFO:teuthology.orchestra.run.smithi079.stdout:2945 write 0x17028d8 thru 0x170837d (0x5aa6 bytes) 2024-09-16T14:08:13.515 INFO:teuthology.orchestra.run.smithi079.stdout:2946 read 0x1ceaa32 thru 0x1cefa19 (0x4fe8 bytes) 2024-09-16T14:08:13.523 INFO:teuthology.orchestra.run.smithi079.stdout:2947 trunc from 0xde5e6fc to 0x98f8673 2024-09-16T14:08:13.538 INFO:teuthology.orchestra.run.smithi079.stdout:2948 writesame 0x23745d2 thru 0x237b82c data_size 0x5(0x725b bytes) 2024-09-16T14:08:13.561 INFO:teuthology.orchestra.run.smithi079.stdout:2953 writesame 0xda80c6e thru 0xda8b287 data_size 0x39(0xa61a bytes) 2024-09-16T14:08:13.562 INFO:teuthology.orchestra.run.smithi079.stdout:2955 punch from 0xd671654 to 0xd6761c7, (0x4b73 bytes) 2024-09-16T14:08:13.563 INFO:teuthology.orchestra.run.smithi079.stdout:2957 write 0x1dd98a thru 0x1ed614 (0xfc8b bytes) 2024-09-16T14:08:13.567 INFO:teuthology.orchestra.run.smithi079.stdout:2959 write 0x12142e thru 0x125642 (0x4215 bytes) 2024-09-16T14:08:13.571 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.571 INFO:teuthology.orchestra.run.smithi079.stdout:2961 read 0x5b9af29 thru 0x5b9d6cd (0x27a5 bytes) 2024-09-16T14:08:13.572 INFO:teuthology.orchestra.run.smithi079.stdout:2964 read 0x1401f3 thru 0x14f134 (0xef42 bytes) 2024-09-16T14:08:13.575 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:13.573+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:13.580 INFO:teuthology.orchestra.run.smithi079.stdout:2965 trunc from 0xda8b288 to 0x92c94c5 2024-09-16T14:08:13.606 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:13.606 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:13.609 INFO:teuthology.orchestra.run.smithi079.stdout:2967 trunc from 0x92c94c5 to 0x2a057c5 2024-09-16T14:08:13.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:13.645 INFO:teuthology.orchestra.run.smithi079.stdout:2969 write 0xa9bf5ee thru 0xa9ce992 (0xf3a5 bytes) 2024-09-16T14:08:13.648 INFO:teuthology.orchestra.run.smithi079.stdout:2971 write 0x85758a8 thru 0x8584179 (0xe8d2 bytes) 2024-09-16T14:08:13.650 INFO:teuthology.orchestra.run.smithi079.stdout:2972 writesame 0x7cc7ab3 thru 0x7cd5ab8 data_size 0x9b(0xe006 bytes) 2024-09-16T14:08:13.657 INFO:teuthology.orchestra.run.smithi079.stdout:2973 write 0x2155b38 thru 0x2162c70 (0xd139 bytes) 2024-09-16T14:08:13.662 INFO:teuthology.orchestra.run.smithi079.stdout:2974 trunc from 0xa9ce993 to 0x942b24a 2024-09-16T14:08:13.675 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:13.673+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:13.677 INFO:teuthology.orchestra.run.smithi079.stdout:2975 write 0x8c1fff thru 0x8cd9d6 (0xb9d8 bytes) 2024-09-16T14:08:13.687 INFO:teuthology.orchestra.run.smithi079.stdout:2979 trunc from 0x942b24a to 0xd67133e 2024-09-16T14:08:13.696 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:13.696 INFO:teuthology.orchestra.run.smithi079.stdout:2982 read 0x6b1e7a9 thru 0x6b267ed (0x8045 bytes) 2024-09-16T14:08:13.697 INFO:teuthology.orchestra.run.smithi079.stdout:2984 write 0x4dc6fa5 thru 0x4dcac11 (0x3c6d bytes) 2024-09-16T14:08:13.700 INFO:teuthology.orchestra.run.smithi079.stdout:2985 write 0xc882eaa thru 0xc88a948 (0x7a9f bytes) 2024-09-16T14:08:13.703 INFO:teuthology.orchestra.run.smithi079.stdout:2986 write 0xad00e8 thru 0xad6d95 (0x6cae bytes) 2024-09-16T14:08:13.706 INFO:teuthology.orchestra.run.smithi079.stdout:2987 write 0x2c69706 thru 0x2c76903 (0xd1fe bytes) 2024-09-16T14:08:13.715 INFO:teuthology.orchestra.run.smithi079.stdout:2989 trunc from 0xd67133e to 0xaa5f497 2024-09-16T14:08:13.730 INFO:teuthology.orchestra.run.smithi079.stdout:2992 punch from 0x6095126 to 0x60978e5, (0x27bf bytes) 2024-09-16T14:08:13.730 INFO:teuthology.orchestra.run.smithi079.stdout:2995 clone 27 order 18 su 32768 sc 8 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838786037999999, 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838776311, 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838752748000001, 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838725509, 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838716095999999, 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838691777000001, 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838672170999999, 2024-09-16T14:08:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838651982999998, 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838628713999999, 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838598464, 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838581712, 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838564448, 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838525612000002, 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:13.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838490784000001, 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:13.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:13.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838454897999998, 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:13.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.837 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838432306000001, 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:13.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:13.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:13.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838403234000001, 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:13.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.848 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:13.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:13.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838384888, 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:13.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:13.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838367549000001, 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:13.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:13.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.838351626000001, 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:13.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:13.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:13.870 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:13.870 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:13.871 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:13.872 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:13.866+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:13.972 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:13.970+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:14.021 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:14.021 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:14.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:14.073 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:14.069+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:14.141 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:14.141 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:14.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:14.173 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:14.169+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:14.257 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:14.257 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:14.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.345770762000001, 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:08:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.345748054000001, 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:08:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:08:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:08:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.345723902, 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.345690960999999, 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:08:14.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.345666277999999, 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:08:14.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:08:14.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:08:14.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:14.285 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.345643049, 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:14.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:08:14.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.343797084999999, 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:14.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.343754221000001, 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:08:14.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:08:14.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.301 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342294602999999, 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:14.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:08:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342278833999998, 2024-09-16T14:08:14.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:14.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:14.313 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342263081999999, 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:14.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:08:14.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342247651000001, 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:08:14.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:08:14.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:08:14.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:08:14.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:08:14.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342232013, 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:14.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:08:14.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:14.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342216874000002, 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:08:14.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342202037, 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:08:14.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342179033000001, 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:14.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:08:14.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:08:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342163807999999, 2024-09-16T14:08:14.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:14.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:08:14.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342148624, 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342133379, 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:08:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.342118221, 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:08:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.370 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:14.371 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-16T14:08:14.371 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:08:14.371 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T14:08:14.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T14:08:14.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:14.373 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:14.369+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:14.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:14.412 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone26 from 0xaa5f497 (overlap 0xaac600) to 0x1113cc 2024-09-16T14:08:14.473 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:14.466+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:14.524 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:14.524 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:14.525 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:14.525 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:14.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:14.567 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:14.573 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:14.569+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:14.601 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:14.632 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:14.636 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:14.636 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:14.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:14.665 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:14.674 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:14.666+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:14.698 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:14.721 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-16T14:08:14.731 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:14.738 INFO:teuthology.orchestra.run.smithi079.stdout:2997 writesame 0x899358 thru 0x89dc63 data_size 0x19(0x490c bytes) 2024-09-16T14:08:14.753 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:14.753 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:14.755 INFO:teuthology.orchestra.run.smithi079.stdout:2999 write 0x355273d thru 0x355e775 (0xc039 bytes) 2024-09-16T14:08:14.756 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:14.756 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:14.756 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:14.757 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:14.757 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:14.757 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:14.757 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:14.757 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:14.757 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:14.761 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:14.761 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:14.763 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:14.766 INFO:teuthology.orchestra.run.smithi079.stdout:3002 writesame 0x6f7cc55 thru 0x6f8bd6b data_size 0x7(0xf117 bytes) 2024-09-16T14:08:14.780 INFO:teuthology.orchestra.run.smithi079.stdout:3003 write 0x648d7ce thru 0x649d3d4 (0xfc07 bytes) 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 73, 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "a22b38a6-94db-470d-bf35-45d7605f3bb0", 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-16T14:06:40.573726+0000", 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-16T14:08:14.080763+0000", 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-16T14:07:53.884253+0000", 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-16T14:06:42.748450+0000", 2024-09-16T14:08:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 5, 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T14:08:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 3, 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "jewel", 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "quincy", 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:52.822826+0000", 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:08:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-16T14:08:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "18", 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:08:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:08:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:08:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-16T14:08:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:58.526935+0000", 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:08:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:08:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "22", 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 22, 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:08:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:08:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:08:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "pool_client.0", 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:07:08.375919+0000", 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:08:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-16T14:08:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "73", 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:08:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 30, 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 73, 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:08:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:08:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a060b161-fd5a-4cfb-bac8-f2acc32d7955", 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 59, 2024-09-16T14:08:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-16T14:08:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/22196", 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/22196", 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6824/22196", 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/22196", 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "f5ba4b39-ceb8-46b3-938e-57249ed84b87", 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 59, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-16T14:08:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/22195", 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/22195", 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/22195", 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/22195", 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "8b657f92-66c9-46a3-a31f-0bd0be414e4e", 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 59, 2024-09-16T14:08:14.820 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.822 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6825/22197", 2024-09-16T14:08:14.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6827/22197", 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6831/22197", 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6829/22197", 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "15937bd7-8964-4140-9f86-b3093c318d1e", 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 0, 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:14.825 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 23, 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 59, 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:08:14.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.827 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.828 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22193 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/22193", 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/22193", 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/22193", 2024-09-16T14:08:14.830 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/22193", 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "exists" 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "714baf80-ade0-4e88-ad4e-5171add1fc6a", 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.242828369140625, 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:14.831 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 14, 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 59, 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6808", 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.832 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6809", 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6810", 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.833 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6811", 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6814", 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.834 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6815", 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6812", 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6813", 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6809/22362", 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6811/22362", 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6815/22362", 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6813/22362", 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e5a21b84-81ab-45da-a081-3b504f5c4f3d", 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 59, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:14.837 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6800", 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6801", 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6802", 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.839 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6803", 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6806", 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6807", 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6804", 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6805", 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6801/22363", 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6803/22363", 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6807/22363", 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6805/22363", 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a009857f-f798-4154-9ff4-f524f60b6c61", 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 59, 2024-09-16T14:08:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6824", 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6825", 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6826", 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6827", 2024-09-16T14:08:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6830", 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6831", 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6828", 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6829", 2024-09-16T14:08:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6825/22364", 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6827/22364", 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6831/22364", 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6829/22364", 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e3506002-ba8b-4251-8ef1-136334574a6d", 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 34, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6816", 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6817", 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6818", 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6819", 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6822", 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6823", 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6820", 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6821", 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6817/22365", 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6819/22365", 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6823/22365", 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6821/22365", 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.104631+0000", 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987403+0000", 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.018134+0000", 2024-09-16T14:08:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987645+0000", 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 59 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:08:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.721507+0000", 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.869796+0000", 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.206576+0000", 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.038098+0000", 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-16T14:08:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-16T14:08:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-16T14:08:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-16T14:08:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-16T14:08:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:14.863 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:14.863 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:14.802+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:14.864 INFO:teuthology.orchestra.run.smithi079.stdout:3005 read 0x19a63ab thru 0x19ad0fe (0x6d54 bytes) 2024-09-16T14:08:14.864 INFO:teuthology.orchestra.run.smithi079.stdout:3006 write 0x596628a thru 0x596d288 (0x6fff bytes) 2024-09-16T14:08:14.865 INFO:tasks.thrashosds.thrasher:j is {'epoch': 73, 'fsid': 'a22b38a6-94db-470d-bf35-45d7605f3bb0', 'created': '2024-09-16T14:06:40.573726+0000', 'modified': '2024-09-16T14:08:14.080763+0000', 'last_up_change': '2024-09-16T14:07:53.884253+0000', 'last_in_change': '2024-09-16T14:06:42.748450+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T14:06:52.822826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T14:06:58.526935+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T14:07:08.375919+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 73, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': 'a060b161-fd5a-4cfb-bac8-f2acc32d7955', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 22196}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 22196}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6824', 'nonce': 22196}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 22196}]}, 'public_addr': '172.21.15.79:6817/22196', 'cluster_addr': '172.21.15.79:6819/22196', 'heartbeat_back_addr': '172.21.15.79:6824/22196', 'heartbeat_front_addr': '172.21.15.79:6821/22196', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f5ba4b39-ceb8-46b3-938e-57249ed84b87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 22195}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 22195}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 22195}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 22195}]}, 'public_addr': '172.21.15.79:6809/22195', 'cluster_addr': '172.21.15.79:6811/22195', 'heartbeat_back_addr': '172.21.15.79:6815/22195', 'heartbeat_front_addr': '172.21.15.79:6813/22195', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8b657f92-66c9-46a3-a31f-0bd0be414e4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6823', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 22197}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 22197}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 22197}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 22197}]}, 'public_addr': '172.21.15.79:6825/22197', 'cluster_addr': '172.21.15.79:6827/22197', 'heartbeat_back_addr': '172.21.15.79:6831/22197', 'heartbeat_front_addr': '172.21.15.79:6829/22197', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '15937bd7-8964-4140-9f86-b3093c318d1e', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 59, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 22193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 22193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 22193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 22193}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 22193}]}, 'public_addr': '172.21.15.79:6801/22193', 'cluster_addr': '172.21.15.79:6803/22193', 'heartbeat_back_addr': '172.21.15.79:6807/22193', 'heartbeat_front_addr': '172.21.15.79:6805/22193', 'state': ['exists']}, {'osd': 4, 'uuid': '714baf80-ade0-4e88-ad4e-5171add1fc6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.242828369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 22362}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 22362}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 22362}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 22362}]}, 'public_addr': '172.21.15.164:6809/22362', 'cluster_addr': '172.21.15.164:6811/22362', 'heartbeat_back_addr': '172.21.15.164:6815/22362', 'heartbeat_front_addr': '172.21.15.164:6813/22362', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e5a21b84-81ab-45da-a081-3b504f5c4f3d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 22363}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 22363}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 22363}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 22363}]}, 'public_addr': '172.21.15.164:6801/22363', 'cluster_addr': '172.21.15.164:6803/22363', 'heartbeat_back_addr': '172.21.15.164:6807/22363', 'heartbeat_front_addr': '172.21.15.164:6805/22363', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a009857f-f798-4154-9ff4-f524f60b6c61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 22364}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 22364}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 22364}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 22364}]}, 'public_addr': '172.21.15.164:6825/22364', 'cluster_addr': '172.21.15.164:6827/22364', 'heartbeat_back_addr': '172.21.15.164:6831/22364', 'heartbeat_front_addr': '172.21.15.164:6829/22364', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e3506002-ba8b-4251-8ef1-136334574a6d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 22365}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 22365}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 22365}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 22365}]}, 'public_addr': '172.21.15.164:6817/22365', 'cluster_addr': '172.21.15.164:6819/22365', 'heartbeat_back_addr': '172.21.15.164:6823/22365', 'heartbeat_front_addr': '172.21.15.164:6821/22365', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.104631+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987403+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.018134+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987645+0000', 'dead_epoch': 59}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.721507+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.869796+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.206576+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.038098+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T14:08:14.866 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:08:14.866 INFO:teuthology.orchestra.run.smithi079.stdout:3009 write 0x9f48a9b thru 0x9f574ca (0xea30 bytes) 2024-09-16T14:08:14.867 INFO:teuthology.orchestra.run.smithi079.stdout:3010 read 0x6da9316 thru 0x6db50ae (0xbd99 bytes) 2024-09-16T14:08:14.867 INFO:teuthology.orchestra.run.smithi079.stdout:3011 read 0x9afd21c thru 0x9b037ed (0x65d2 bytes) 2024-09-16T14:08:14.867 INFO:teuthology.orchestra.run.smithi079.stdout:3014 writesame 0x513c272 thru 0x5149323 data_size 0x1(0xd0b2 bytes) 2024-09-16T14:08:14.867 INFO:teuthology.orchestra.run.smithi079.stdout:3015 write 0x9eafb4c thru 0x9eb803f (0x84f4 bytes) 2024-09-16T14:08:14.869 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:14.869 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:14.875 INFO:teuthology.orchestra.run.smithi079.stdout:3017 punch from 0x52450b to 0x526d96, (0x288b bytes) 2024-09-16T14:08:14.875 INFO:teuthology.orchestra.run.smithi079.stdout:3018 write 0x2595599 thru 0x259a136 (0x4b9e bytes) 2024-09-16T14:08:14.876 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:14.870+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:14.879 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:14.892 INFO:teuthology.orchestra.run.smithi079.stdout:3019 trunc from 0xaa5f497 to 0x68fa72d 2024-09-16T14:08:14.926 INFO:teuthology.orchestra.run.smithi079.stdout:3020 read 0x5594d30 thru 0x55a31c8 (0xe499 bytes) 2024-09-16T14:08:14.945 INFO:teuthology.orchestra.run.smithi079.stdout:3023 write 0x92d66df thru 0x92e4924 (0xe246 bytes) 2024-09-16T14:08:14.965 INFO:teuthology.orchestra.run.smithi079.stdout:3024 write 0x9cb832c thru 0x9cb9ef6 (0x1bcb bytes) 2024-09-16T14:08:14.976 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:14.970+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:14.980 INFO:teuthology.orchestra.run.smithi079.stdout:3025 trunc from 0x9cb9ef7 to 0x449990d 2024-09-16T14:08:14.983 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:14.983 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:14.994 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:15.026 INFO:teuthology.orchestra.run.smithi079.stdout:3026 write 0x677cb39 thru 0x677ec01 (0x20c9 bytes) 2024-09-16T14:08:15.030 INFO:teuthology.orchestra.run.smithi079.stdout:3027 read 0x6fa58c thru 0x6fd2fd (0x2d72 bytes) 2024-09-16T14:08:15.031 INFO:teuthology.orchestra.run.smithi079.stdout:3030 write 0x3ade3fc thru 0x3ae53a6 (0x6fab bytes) 2024-09-16T14:08:15.045 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:15.070 INFO:teuthology.orchestra.run.smithi079.stdout:3032 writesame 0xd2b6971 thru 0xd2b7741 data_size 0x9(0xdd1 bytes) 2024-09-16T14:08:15.070 INFO:teuthology.orchestra.run.smithi079.stdout:3033 clone 28 order 20 su 65536 sc 4 2024-09-16T14:08:15.077 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:15.073+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:15.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:15.151 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:15.178 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:15.170+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:15.186 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:08:15.188 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:08:15.200 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-16T14:08:15.097790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":818807285,"num_objects":2157,"num_object_clones":497,"num_object_copies":4314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":668,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2157,"num_whiteouts":550,"num_read":171278,"num_read_kb":366444,"num_write":17001,"num_write_kb":220554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13032,"ondisk_log_size":13032,"up":67,"acting":67,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2489516,"kb_used_data":35740,"kb_used_omap":13,"kb_used_meta":2453618,"kb_avail":752485204,"statfs":{"total":773094113280,"available":770544848896,"internally_reserved":0,"allocated":36597760,"data_stored":271268189,"data_compressed":5224901,"data_compressed_allocated":19566592,"data_compressed_original":262087309,"omap_allocated":13632,"internal_metadata":2512505536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":189335750,"num_objects":122,"num_object_clones":49,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":41,"num_read":11853,"num_read_kb":132900,"num_write":834,"num_write_kb":70336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.312573"},"pg_stats":[{"pgid":"3.1f","version":"70'1511","reported_seq":5722,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:10.889908+0000","last_change":"2024-09-16T14:07:11.338271+0000","last_active":"2024-09-16T14:08:10.889908+0000","last_peered":"2024-09-16T14:08:10.889908+0000","last_clean":"2024-09-16T14:08:10.889908+0000","last_became_active":"2024-09-16T14:07:09.540900+0000","last_became_peered":"2024-09-16T14:07:09.540900+0000","last_unstale":"2024-09-16T14:08:10.889908+0000","last_undegraded":"2024-09-16T14:08:10.889908+0000","last_fullsized":"2024-09-16T14:08:10.889908+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1511,"log_dups_size":0,"ondisk_log_size":1511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:59.820596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233770000000001,"stat_sum":{"num_bytes":17016636,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":11,"num_read":13272,"num_read_kb":14126,"num_write":2645,"num_write_kb":5886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"70'518","reported_seq":1646,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.260323+0000","last_change":"2024-09-16T14:07:11.337654+0000","last_active":"2024-09-16T14:08:11.260323+0000","last_peered":"2024-09-16T14:08:11.260323+0000","last_clean":"2024-09-16T14:08:11.260323+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:08:11.260323+0000","last_undegraded":"2024-09-16T14:08:11.260323+0000","last_fullsized":"2024-09-16T14:08:11.260323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060911299999999999,"stat_sum":{"num_bytes":23963692,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":14,"num_read":2830,"num_read_kb":16035,"num_write":753,"num_write_kb":7069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"70'319","reported_seq":4325,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:11.243081+0000","last_change":"2024-09-16T14:07:54.912706+0000","last_active":"2024-09-16T14:08:11.243081+0000","last_peered":"2024-09-16T14:08:11.243081+0000","last_clean":"2024-09-16T14:07:46.901101+0000","last_became_active":"2024-09-16T14:07:54.912706+0000","last_became_peered":"2024-09-16T14:07:54.912706+0000","last_unstale":"2024-09-16T14:08:11.243081+0000","last_undegraded":"2024-09-16T14:07:54.911389+0000","last_fullsized":"2024-09-16T14:07:54.911287+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:02.876106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072576699999999999,"stat_sum":{"num_bytes":30084243,"num_objects":79,"num_object_clones":18,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":17,"num_read":5326,"num_read_kb":15550,"num_write":500,"num_write_kb":10591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":79}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"70'609","reported_seq":12011,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.181055+0000","last_change":"2024-09-16T14:07:11.340416+0000","last_active":"2024-09-16T14:08:11.181055+0000","last_peered":"2024-09-16T14:08:11.181055+0000","last_clean":"2024-09-16T14:08:11.181055+0000","last_became_active":"2024-09-16T14:07:09.551657+0000","last_became_peered":"2024-09-16T14:07:09.551657+0000","last_unstale":"2024-09-16T14:08:11.181055+0000","last_undegraded":"2024-09-16T14:08:11.181055+0000","last_fullsized":"2024-09-16T14:08:11.181055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:49.635426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064309800000000004,"stat_sum":{"num_bytes":11605791,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":13503,"num_read_kb":11643,"num_write":901,"num_write_kb":2731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"69'327","reported_seq":2463,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.253100+0000","last_change":"2024-09-16T14:07:22.350217+0000","last_active":"2024-09-16T14:08:11.253100+0000","last_peered":"2024-09-16T14:08:11.253100+0000","last_clean":"2024-09-16T14:08:11.253100+0000","last_became_active":"2024-09-16T14:07:22.349546+0000","last_became_peered":"2024-09-16T14:07:22.349546+0000","last_unstale":"2024-09-16T14:08:11.253100+0000","last_undegraded":"2024-09-16T14:08:11.253100+0000","last_fullsized":"2024-09-16T14:08:11.253100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:30.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021275310000000002,"stat_sum":{"num_bytes":14997219,"num_objects":70,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":18,"num_read":3807,"num_read_kb":9836,"num_write":392,"num_write_kb":4439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"70'351","reported_seq":3687,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.369558+0000","last_change":"2024-09-16T14:07:11.340816+0000","last_active":"2024-09-16T14:08:11.369558+0000","last_peered":"2024-09-16T14:08:11.369558+0000","last_clean":"2024-09-16T14:08:11.369558+0000","last_became_active":"2024-09-16T14:07:09.540237+0000","last_became_peered":"2024-09-16T14:07:09.540237+0000","last_unstale":"2024-09-16T14:08:11.369558+0000","last_undegraded":"2024-09-16T14:08:11.369558+0000","last_fullsized":"2024-09-16T14:08:11.369558+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:38.147579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099540499999999999,"stat_sum":{"num_bytes":17596372,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":14,"num_read":4171,"num_read_kb":14284,"num_write":384,"num_write_kb":18756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"70'287","reported_seq":849,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.248990+0000","last_change":"2024-09-16T14:07:22.346713+0000","last_active":"2024-09-16T14:08:11.248990+0000","last_peered":"2024-09-16T14:08:11.248990+0000","last_clean":"2024-09-16T14:08:11.248990+0000","last_became_active":"2024-09-16T14:07:22.346031+0000","last_became_peered":"2024-09-16T14:07:22.346031+0000","last_unstale":"2024-09-16T14:08:11.248990+0000","last_undegraded":"2024-09-16T14:08:11.248990+0000","last_fullsized":"2024-09-16T14:08:11.248990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:39.265638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003350142,"stat_sum":{"num_bytes":27255081,"num_objects":54,"num_object_clones":10,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":13,"num_read":894,"num_read_kb":8387,"num_write":337,"num_write_kb":15115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"70'308","reported_seq":4178,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.364934+0000","last_change":"2024-09-16T14:07:11.338172+0000","last_active":"2024-09-16T14:08:11.364934+0000","last_peered":"2024-09-16T14:08:11.364934+0000","last_clean":"2024-09-16T14:08:11.364934+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:08:11.364934+0000","last_undegraded":"2024-09-16T14:08:11.364934+0000","last_fullsized":"2024-09-16T14:08:11.364934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148865,"stat_sum":{"num_bytes":24980849,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":12,"num_read":4870,"num_read_kb":6535,"num_write":330,"num_write_kb":5427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"70'756","reported_seq":2836,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:11.180306+0000","last_change":"2024-09-16T14:07:54.913217+0000","last_active":"2024-09-16T14:08:11.180306+0000","last_peered":"2024-09-16T14:08:11.180306+0000","last_clean":"2024-09-16T14:07:52.630085+0000","last_became_active":"2024-09-16T14:07:54.913217+0000","last_became_peered":"2024-09-16T14:07:54.913217+0000","last_unstale":"2024-09-16T14:08:11.180306+0000","last_undegraded":"2024-09-16T14:07:54.912301+0000","last_fullsized":"2024-09-16T14:07:54.912203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:45.157064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215788,"stat_sum":{"num_bytes":8149847,"num_objects":59,"num_object_clones":14,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":59,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":17,"num_read":5228,"num_read_kb":5606,"num_write":1214,"num_write_kb":2697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":59}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"70'308","reported_seq":3243,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:11.267009+0000","last_change":"2024-09-16T14:07:54.909454+0000","last_active":"2024-09-16T14:08:11.267009+0000","last_peered":"2024-09-16T14:08:11.267009+0000","last_clean":"2024-09-16T14:07:51.872763+0000","last_became_active":"2024-09-16T14:07:54.909454+0000","last_became_peered":"2024-09-16T14:07:54.909454+0000","last_unstale":"2024-09-16T14:08:11.267009+0000","last_undegraded":"2024-09-16T14:07:54.907634+0000","last_fullsized":"2024-09-16T14:07:54.907532+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:22.254267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500454,"stat_sum":{"num_bytes":17994247,"num_objects":75,"num_object_clones":18,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":75,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":24,"num_read":3438,"num_read_kb":9830,"num_write":324,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":75}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"69'241","reported_seq":3343,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.254065+0000","last_change":"2024-09-16T14:07:11.343447+0000","last_active":"2024-09-16T14:08:11.254065+0000","last_peered":"2024-09-16T14:08:11.254065+0000","last_clean":"2024-09-16T14:08:11.254065+0000","last_became_active":"2024-09-16T14:07:09.549545+0000","last_became_peered":"2024-09-16T14:07:09.549545+0000","last_unstale":"2024-09-16T14:08:11.254065+0000","last_undegraded":"2024-09-16T14:08:11.254065+0000","last_fullsized":"2024-09-16T14:08:11.254065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:27.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012153120000000001,"stat_sum":{"num_bytes":29539505,"num_objects":75,"num_object_clones":25,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":24,"num_read":3155,"num_read_kb":10736,"num_write":269,"num_write_kb":3814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"66'272","reported_seq":1485,"reported_epoch":69,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:08.042775+0000","last_change":"2024-09-16T14:07:54.908813+0000","last_active":"2024-09-16T14:08:08.042775+0000","last_peered":"2024-09-16T14:08:08.042775+0000","last_clean":"2024-09-16T14:07:51.884715+0000","last_became_active":"2024-09-16T14:07:54.908813+0000","last_became_peered":"2024-09-16T14:07:54.908813+0000","last_unstale":"2024-09-16T14:08:08.042775+0000","last_undegraded":"2024-09-16T14:07:54.907494+0000","last_fullsized":"2024-09-16T14:07:54.907370+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:57.481804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009984429999999999,"stat_sum":{"num_bytes":22979007,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":63,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":13,"num_read":2255,"num_read_kb":4691,"num_write":276,"num_write_kb":2760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":63}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+undersized","last_fresh":"2024-09-16T14:08:08.139142+0000","last_change":"2024-09-16T14:07:54.912496+0000","last_active":"2024-09-16T14:08:08.139142+0000","last_peered":"2024-09-16T14:08:08.139142+0000","last_clean":"2024-09-16T14:06:59.453408+0000","last_became_active":"2024-09-16T14:07:54.912496+0000","last_became_peered":"2024-09-16T14:07:54.912496+0000","last_unstale":"2024-09-16T14:08:08.139142+0000","last_undegraded":"2024-09-16T14:08:08.139142+0000","last_fullsized":"2024-09-16T14:07:54.911560+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:44.667451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"69'457","reported_seq":2543,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:10.629433+0000","last_change":"2024-09-16T14:07:11.370476+0000","last_active":"2024-09-16T14:08:10.629433+0000","last_peered":"2024-09-16T14:08:10.629433+0000","last_clean":"2024-09-16T14:08:10.629433+0000","last_became_active":"2024-09-16T14:07:09.544524+0000","last_became_peered":"2024-09-16T14:07:09.544524+0000","last_unstale":"2024-09-16T14:08:10.629433+0000","last_undegraded":"2024-09-16T14:08:10.629433+0000","last_fullsized":"2024-09-16T14:08:10.629433+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:59.087380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032490359999999999,"stat_sum":{"num_bytes":26464903,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":12,"num_read":3558,"num_read_kb":5955,"num_write":582,"num_write_kb":2847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T14:08:08.139027+0000","last_change":"2024-09-16T14:07:22.350069+0000","last_active":"2024-09-16T14:08:08.139027+0000","last_peered":"2024-09-16T14:08:08.139027+0000","last_clean":"2024-09-16T14:08:08.139027+0000","last_became_active":"2024-09-16T14:07:22.349757+0000","last_became_peered":"2024-09-16T14:07:22.349757+0000","last_unstale":"2024-09-16T14:08:08.139027+0000","last_undegraded":"2024-09-16T14:08:08.139027+0000","last_fullsized":"2024-09-16T14:08:08.139027+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:57.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"70'481","reported_seq":5534,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.358598+0000","last_change":"2024-09-16T14:07:22.351951+0000","last_active":"2024-09-16T14:08:11.358598+0000","last_peered":"2024-09-16T14:08:11.358598+0000","last_clean":"2024-09-16T14:08:11.358598+0000","last_became_active":"2024-09-16T14:07:22.351394+0000","last_became_peered":"2024-09-16T14:07:22.351394+0000","last_unstale":"2024-09-16T14:08:11.358598+0000","last_undegraded":"2024-09-16T14:08:11.358598+0000","last_fullsized":"2024-09-16T14:08:11.358598+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:55.945287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027484229999999998,"stat_sum":{"num_bytes":67690610,"num_objects":71,"num_object_clones":16,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":19,"num_read":6892,"num_read_kb":27503,"num_write":630,"num_write_kb":15654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+undersized","last_fresh":"2024-09-16T14:08:08.125801+0000","last_change":"2024-09-16T14:07:54.912695+0000","last_active":"2024-09-16T14:08:08.125801+0000","last_peered":"2024-09-16T14:08:08.125801+0000","last_clean":"2024-09-16T14:06:59.453408+0000","last_became_active":"2024-09-16T14:07:54.912695+0000","last_became_peered":"2024-09-16T14:07:54.912695+0000","last_unstale":"2024-09-16T14:08:08.125801+0000","last_undegraded":"2024-09-16T14:08:08.125801+0000","last_fullsized":"2024-09-16T14:07:54.910709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:49.193562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"70'294","reported_seq":2269,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.259326+0000","last_change":"2024-09-16T14:07:11.547417+0000","last_active":"2024-09-16T14:08:11.259326+0000","last_peered":"2024-09-16T14:08:11.259326+0000","last_clean":"2024-09-16T14:08:11.259326+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:08:11.259326+0000","last_undegraded":"2024-09-16T14:08:11.259326+0000","last_fullsized":"2024-09-16T14:08:11.259326+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097608199999999997,"stat_sum":{"num_bytes":11334294,"num_objects":68,"num_object_clones":17,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":21,"num_read":2391,"num_read_kb":9569,"num_write":331,"num_write_kb":3345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T14:08:08.046529+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:08:08.046529+0000","last_peered":"2024-09-16T14:08:08.046529+0000","last_clean":"2024-09-16T14:08:08.046529+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:08:08.046529+0000","last_undegraded":"2024-09-16T14:08:08.046529+0000","last_fullsized":"2024-09-16T14:08:08.046529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"70'418","reported_seq":8325,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.259833+0000","last_change":"2024-09-16T14:07:11.361331+0000","last_active":"2024-09-16T14:08:11.259833+0000","last_peered":"2024-09-16T14:08:11.259833+0000","last_clean":"2024-09-16T14:08:11.259833+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:08:11.259833+0000","last_undegraded":"2024-09-16T14:08:11.259833+0000","last_fullsized":"2024-09-16T14:08:11.259833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062220600000000004,"stat_sum":{"num_bytes":23573877,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":20,"num_read":8926,"num_read_kb":10512,"num_write":531,"num_write_kb":2415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"70'517","reported_seq":3090,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.259766+0000","last_change":"2024-09-16T14:07:11.342816+0000","last_active":"2024-09-16T14:08:11.259766+0000","last_peered":"2024-09-16T14:08:11.259766+0000","last_clean":"2024-09-16T14:08:11.259766+0000","last_became_active":"2024-09-16T14:07:09.549842+0000","last_became_peered":"2024-09-16T14:07:09.549842+0000","last_unstale":"2024-09-16T14:08:11.259766+0000","last_undegraded":"2024-09-16T14:08:11.259766+0000","last_fullsized":"2024-09-16T14:08:11.259766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:47.724999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014467779999999999,"stat_sum":{"num_bytes":22186190,"num_objects":77,"num_object_clones":16,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":24,"num_read":5808,"num_read_kb":11714,"num_write":549,"num_write_kb":5317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":30,"reported_epoch":69,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:08.057444+0000","last_change":"2024-09-16T14:07:54.909243+0000","last_active":"2024-09-16T14:08:08.057444+0000","last_peered":"2024-09-16T14:08:08.057444+0000","last_clean":"2024-09-16T14:07:02.502232+0000","last_became_active":"2024-09-16T14:07:54.909243+0000","last_became_peered":"2024-09-16T14:07:54.909243+0000","last_unstale":"2024-09-16T14:08:08.057444+0000","last_undegraded":"2024-09-16T14:07:54.906893+0000","last_fullsized":"2024-09-16T14:07:54.906672+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:49.832533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":1}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"70'706","reported_seq":3542,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.376395+0000","last_change":"2024-09-16T14:07:11.337642+0000","last_active":"2024-09-16T14:08:11.376395+0000","last_peered":"2024-09-16T14:08:11.376395+0000","last_clean":"2024-09-16T14:08:11.376395+0000","last_became_active":"2024-09-16T14:07:09.543015+0000","last_became_peered":"2024-09-16T14:07:09.543015+0000","last_unstale":"2024-09-16T14:08:11.376395+0000","last_undegraded":"2024-09-16T14:08:11.376395+0000","last_fullsized":"2024-09-16T14:08:11.376395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:22:22.438444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076943300000000003,"stat_sum":{"num_bytes":47539218,"num_objects":88,"num_object_clones":23,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":24,"num_read":5298,"num_read_kb":15365,"num_write":1073,"num_write_kb":6678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T14:08:08.052027+0000","last_change":"2024-09-16T14:07:04.121594+0000","last_active":"2024-09-16T14:08:08.052027+0000","last_peered":"2024-09-16T14:08:08.052027+0000","last_clean":"2024-09-16T14:08:08.052027+0000","last_became_active":"2024-09-16T14:07:00.475232+0000","last_became_peered":"2024-09-16T14:07:00.475232+0000","last_unstale":"2024-09-16T14:08:08.052027+0000","last_undegraded":"2024-09-16T14:08:08.052027+0000","last_fullsized":"2024-09-16T14:08:08.052027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.457456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068024199999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"70'422","reported_seq":5287,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.259990+0000","last_change":"2024-09-16T14:07:11.547270+0000","last_active":"2024-09-16T14:08:11.259990+0000","last_peered":"2024-09-16T14:08:11.259990+0000","last_clean":"2024-09-16T14:08:11.259990+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:08:11.259990+0000","last_undegraded":"2024-09-16T14:08:11.259990+0000","last_fullsized":"2024-09-16T14:08:11.259990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085373199999999995,"stat_sum":{"num_bytes":27236681,"num_objects":68,"num_object_clones":15,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":18,"num_read":6256,"num_read_kb":13142,"num_write":572,"num_write_kb":9367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+undersized","last_fresh":"2024-09-16T14:08:08.046300+0000","last_change":"2024-09-16T14:07:54.912727+0000","last_active":"2024-09-16T14:08:08.046300+0000","last_peered":"2024-09-16T14:08:08.046300+0000","last_clean":"2024-09-16T14:06:59.453408+0000","last_became_active":"2024-09-16T14:07:54.912727+0000","last_became_peered":"2024-09-16T14:07:54.912727+0000","last_unstale":"2024-09-16T14:08:08.046300+0000","last_undegraded":"2024-09-16T14:08:08.046300+0000","last_fullsized":"2024-09-16T14:07:54.911883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:23.430177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"69'270","reported_seq":9659,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.260161+0000","last_change":"2024-09-16T14:07:22.350954+0000","last_active":"2024-09-16T14:08:11.260161+0000","last_peered":"2024-09-16T14:08:11.260161+0000","last_clean":"2024-09-16T14:08:11.260161+0000","last_became_active":"2024-09-16T14:07:22.350024+0000","last_became_peered":"2024-09-16T14:07:22.350024+0000","last_unstale":"2024-09-16T14:08:11.260161+0000","last_undegraded":"2024-09-16T14:08:11.260161+0000","last_fullsized":"2024-09-16T14:08:11.260161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:07.634077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002704522,"stat_sum":{"num_bytes":17809652,"num_objects":62,"num_object_clones":18,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":12,"num_read":9450,"num_read_kb":13826,"num_write":236,"num_write_kb":2954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":23,"reported_epoch":69,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:08.042299+0000","last_change":"2024-09-16T14:07:54.908825+0000","last_active":"2024-09-16T14:08:08.042299+0000","last_peered":"2024-09-16T14:08:08.042299+0000","last_clean":"2024-09-16T14:06:59.453408+0000","last_became_active":"2024-09-16T14:07:54.908825+0000","last_became_peered":"2024-09-16T14:07:54.908825+0000","last_unstale":"2024-09-16T14:08:08.042299+0000","last_undegraded":"2024-09-16T14:07:54.907416+0000","last_fullsized":"2024-09-16T14:07:54.907263+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:10.816515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":151,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T14:08:08.041470+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:08:08.041470+0000","last_peered":"2024-09-16T14:08:08.041470+0000","last_clean":"2024-09-16T14:08:08.041470+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:08:08.041470+0000","last_undegraded":"2024-09-16T14:08:08.041470+0000","last_fullsized":"2024-09-16T14:08:08.041470+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"70'260","reported_seq":1737,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:10.383064+0000","last_change":"2024-09-16T14:07:22.352613+0000","last_active":"2024-09-16T14:08:10.383064+0000","last_peered":"2024-09-16T14:08:10.383064+0000","last_clean":"2024-09-16T14:08:10.383064+0000","last_became_active":"2024-09-16T14:07:22.351412+0000","last_became_peered":"2024-09-16T14:07:22.351412+0000","last_unstale":"2024-09-16T14:08:10.383064+0000","last_undegraded":"2024-09-16T14:08:10.383064+0000","last_fullsized":"2024-09-16T14:08:10.383064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:21.471879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022177080000000001,"stat_sum":{"num_bytes":11762822,"num_objects":78,"num_object_clones":16,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":22,"num_read":1601,"num_read_kb":3187,"num_write":265,"num_write_kb":2162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"70'247","reported_seq":4377,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:10.395796+0000","last_change":"2024-09-16T14:07:54.911452+0000","last_active":"2024-09-16T14:08:10.395796+0000","last_peered":"2024-09-16T14:08:10.395796+0000","last_clean":"2024-09-16T14:07:51.884586+0000","last_became_active":"2024-09-16T14:07:54.911452+0000","last_became_peered":"2024-09-16T14:07:54.911452+0000","last_unstale":"2024-09-16T14:08:10.395796+0000","last_undegraded":"2024-09-16T14:07:54.910628+0000","last_fullsized":"2024-09-16T14:07:54.910471+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:02.509811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339700000000001,"stat_sum":{"num_bytes":17266185,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":20,"num_read":4205,"num_read_kb":9064,"num_write":252,"num_write_kb":4117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":70}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"70'451","reported_seq":4752,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:11.342575+0000","last_change":"2024-09-16T14:07:54.906030+0000","last_active":"2024-09-16T14:08:11.342575+0000","last_peered":"2024-09-16T14:08:11.342575+0000","last_clean":"2024-09-16T14:07:51.997431+0000","last_became_active":"2024-09-16T14:07:54.906030+0000","last_became_peered":"2024-09-16T14:07:54.906030+0000","last_unstale":"2024-09-16T14:08:11.342575+0000","last_undegraded":"2024-09-16T14:07:54.905128+0000","last_fullsized":"2024-09-16T14:07:54.905054+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:00.315576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065751299999999998,"stat_sum":{"num_bytes":59964871,"num_objects":88,"num_object_clones":22,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":88,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":23,"num_read":5681,"num_read_kb":8487,"num_write":473,"num_write_kb":14075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":88}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"70'340","reported_seq":1855,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.253561+0000","last_change":"2024-09-16T14:07:11.377884+0000","last_active":"2024-09-16T14:08:11.253561+0000","last_peered":"2024-09-16T14:08:11.253561+0000","last_clean":"2024-09-16T14:08:11.253561+0000","last_became_active":"2024-09-16T14:07:09.544775+0000","last_became_peered":"2024-09-16T14:07:09.544775+0000","last_unstale":"2024-09-16T14:08:11.253561+0000","last_undegraded":"2024-09-16T14:08:11.253561+0000","last_fullsized":"2024-09-16T14:08:11.253561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:56:16.336123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056925200000000002,"stat_sum":{"num_bytes":13056844,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":18,"num_read":2980,"num_read_kb":11008,"num_write":389,"num_write_kb":2501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"69'245","reported_seq":4664,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.248954+0000","last_change":"2024-09-16T14:07:11.343448+0000","last_active":"2024-09-16T14:08:11.248954+0000","last_peered":"2024-09-16T14:08:11.248954+0000","last_clean":"2024-09-16T14:08:11.248954+0000","last_became_active":"2024-09-16T14:07:09.553360+0000","last_became_peered":"2024-09-16T14:07:09.553360+0000","last_unstale":"2024-09-16T14:08:11.248954+0000","last_undegraded":"2024-09-16T14:08:11.248954+0000","last_fullsized":"2024-09-16T14:08:11.248954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:28.854037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017881959999999999,"stat_sum":{"num_bytes":23407892,"num_objects":75,"num_object_clones":17,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":26,"num_read":4476,"num_read_kb":9027,"num_write":215,"num_write_kb":2318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"69'192","reported_seq":7188,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:11.256072+0000","last_change":"2024-09-16T14:07:54.906785+0000","last_active":"2024-09-16T14:08:11.256072+0000","last_peered":"2024-09-16T14:08:11.256072+0000","last_clean":"2024-09-16T14:07:52.488933+0000","last_became_active":"2024-09-16T14:07:54.906785+0000","last_became_peered":"2024-09-16T14:07:54.906785+0000","last_unstale":"2024-09-16T14:08:11.256072+0000","last_undegraded":"2024-09-16T14:07:54.905143+0000","last_fullsized":"2024-09-16T14:07:54.905035+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:01.876472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074836900000000003,"stat_sum":{"num_bytes":26233595,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":14,"num_read":6720,"num_read_kb":14462,"num_write":162,"num_write_kb":2796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":57}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"70'185","reported_seq":2260,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:10.758849+0000","last_change":"2024-09-16T14:07:11.362189+0000","last_active":"2024-09-16T14:08:10.758849+0000","last_peered":"2024-09-16T14:08:10.758849+0000","last_clean":"2024-09-16T14:08:10.758849+0000","last_became_active":"2024-09-16T14:07:09.548973+0000","last_became_peered":"2024-09-16T14:07:09.548973+0000","last_unstale":"2024-09-16T14:08:10.758849+0000","last_undegraded":"2024-09-16T14:08:10.758849+0000","last_fullsized":"2024-09-16T14:08:10.758849+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:17.345008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013505660000000001,"stat_sum":{"num_bytes":13035444,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":13,"num_read":2779,"num_read_kb":2057,"num_write":153,"num_write_kb":1577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"70'467","reported_seq":4128,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.397193+0000","last_change":"2024-09-16T14:07:11.342735+0000","last_active":"2024-09-16T14:08:11.397193+0000","last_peered":"2024-09-16T14:08:11.397193+0000","last_clean":"2024-09-16T14:08:11.397193+0000","last_became_active":"2024-09-16T14:07:09.559054+0000","last_became_peered":"2024-09-16T14:07:09.559054+0000","last_unstale":"2024-09-16T14:08:11.397193+0000","last_undegraded":"2024-09-16T14:08:11.397193+0000","last_fullsized":"2024-09-16T14:08:11.397193+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.926445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372222,"stat_sum":{"num_bytes":38462700,"num_objects":69,"num_object_clones":13,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":14,"num_read":6262,"num_read_kb":13395,"num_write":580,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"70'254","reported_seq":10063,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-16T14:08:11.252489+0000","last_change":"2024-09-16T14:07:11.340733+0000","last_active":"2024-09-16T14:08:11.252489+0000","last_peered":"2024-09-16T14:08:11.252489+0000","last_clean":"2024-09-16T14:08:11.252489+0000","last_became_active":"2024-09-16T14:07:09.540341+0000","last_became_peered":"2024-09-16T14:07:09.540341+0000","last_unstale":"2024-09-16T14:08:11.252489+0000","last_undegraded":"2024-09-16T14:08:11.252489+0000","last_fullsized":"2024-09-16T14:08:11.252489+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:18.953079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012161470000000001,"stat_sum":{"num_bytes":26701788,"num_objects":65,"num_object_clones":16,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":16,"num_read":10133,"num_read_kb":11383,"num_write":462,"num_write_kb":11649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"70'385","reported_seq":3989,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:11.260286+0000","last_change":"2024-09-16T14:07:54.908140+0000","last_active":"2024-09-16T14:08:11.260286+0000","last_peered":"2024-09-16T14:08:11.260286+0000","last_clean":"2024-09-16T14:07:52.309054+0000","last_became_active":"2024-09-16T14:07:54.908140+0000","last_became_peered":"2024-09-16T14:07:54.908140+0000","last_unstale":"2024-09-16T14:08:11.260286+0000","last_undegraded":"2024-09-16T14:07:54.907159+0000","last_fullsized":"2024-09-16T14:07:54.907059+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:24.252554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037127800000000001,"stat_sum":{"num_bytes":17415612,"num_objects":50,"num_object_clones":10,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":10,"num_read":4507,"num_read_kb":10664,"num_write":493,"num_write_kb":6493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":50}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"70'306","reported_seq":3418,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:11.256061+0000","last_change":"2024-09-16T14:07:54.907641+0000","last_active":"2024-09-16T14:08:11.256061+0000","last_peered":"2024-09-16T14:08:11.256061+0000","last_clean":"2024-09-16T14:07:52.494320+0000","last_became_active":"2024-09-16T14:07:54.907641+0000","last_became_peered":"2024-09-16T14:07:54.907641+0000","last_unstale":"2024-09-16T14:08:11.256061+0000","last_undegraded":"2024-09-16T14:07:54.906665+0000","last_fullsized":"2024-09-16T14:07:54.906526+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:57.481804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012662420000000001,"stat_sum":{"num_bytes":30411350,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":14,"num_read":4640,"num_read_kb":27320,"num_write":341,"num_write_kb":20679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":58}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"70'253","reported_seq":5517,"reported_epoch":70,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:08:11.372210+0000","last_change":"2024-09-16T14:07:54.912170+0000","last_active":"2024-09-16T14:08:11.372210+0000","last_peered":"2024-09-16T14:08:11.372210+0000","last_clean":"2024-09-16T14:07:51.884466+0000","last_became_active":"2024-09-16T14:07:54.912170+0000","last_became_peered":"2024-09-16T14:07:54.912170+0000","last_unstale":"2024-09-16T14:08:11.372210+0000","last_undegraded":"2024-09-16T14:07:54.911370+0000","last_fullsized":"2024-09-16T14:07:54.911226+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:40.671856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066482800000000001,"stat_sum":{"num_bytes":50499881,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":67,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":15,"num_read":5900,"num_read_kb":11490,"num_write":284,"num_write_kb":13330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":67}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":818216898,"num_objects":2153,"num_object_clones":497,"num_object_copies":4306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":666,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2153,"num_whiteouts":550,"num_read":171212,"num_read_kb":366389,"num_write":16898,"num_write_kb":218924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32022528,"data_stored":292205626,"data_compressed":5391376,"data_compressed_allocated":21286912,"data_compressed_original":285828373,"omap_allocated":0,"internal_metadata":0},"log_size":12957,"ondisk_log_size":12957,"up":54,"acting":54,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":11,"acting":11,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308716,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063124,"statfs":{"total":96636764160,"available":96320638976,"internally_reserved":0,"allocated":3067904,"data_stored":19564956,"data_compressed":357688,"data_compressed_allocated":1261568,"data_compressed_original":18593858,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":0.39900000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.38},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.44800000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.626},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.432},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.65100000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315456,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94056384,"statfs":{"total":96636764160,"available":96313737216,"internally_reserved":0,"allocated":5758976,"data_stored":52776850,"data_compressed":849341,"data_compressed_allocated":3567616,"data_compressed_original":51630770,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.88},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.85499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.96899999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.622},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.75}]},{"osd":5,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.034},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.999}]}]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306708,"kb_used_data":5196,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94065132,"statfs":{"total":96636764160,"available":96322695168,"internally_reserved":0,"allocated":5320704,"data_stored":46365285,"data_compressed":754264,"data_compressed_allocated":3076096,"data_compressed_original":45133691,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.58699999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.52600000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.501}]},{"osd":6,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.752}]}]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312696,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059144,"statfs":{"total":96636764160,"available":96316563456,"internally_reserved":0,"allocated":2932736,"data_stored":12159821,"data_compressed":563604,"data_compressed_allocated":1290240,"data_compressed_original":11292164,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314332,"kb_used_data":4496,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94057508,"statfs":{"total":96636764160,"available":96314888192,"internally_reserved":0,"allocated":4603904,"data_stored":36099213,"data_compressed":558602,"data_compressed_allocated":2326528,"data_compressed_original":34871650,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.5},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54800000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.86199999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.752},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.91000000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.95799999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302092,"kb_used_data":4740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94069748,"statfs":{"total":96636764160,"available":96327421952,"internally_reserved":0,"allocated":4853760,"data_stored":30177249,"data_compressed":766113,"data_compressed_allocated":2580480,"data_compressed_original":28961493,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":0.68500000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.375}]},{"osd":4,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.054}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.94799999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.97799999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315344,"kb_used_data":5488,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94056496,"statfs":{"total":96636764160,"available":96313851904,"internally_reserved":0,"allocated":5619712,"data_stored":40951356,"data_compressed":832024,"data_compressed_allocated":3108864,"data_compressed_original":39543368,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.623},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.51600000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":1.222}]},{"osd":5,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.012},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.874}]},{"osd":6,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.077},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.92000000000000004}]},{"osd":7,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.05},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.1539999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314172,"kb_used_data":4336,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94057668,"statfs":{"total":96636764160,"available":96315052032,"internally_reserved":0,"allocated":4440064,"data_stored":33173459,"data_compressed":543265,"data_compressed_allocated":2355200,"data_compressed_original":32060315,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.622}]},{"osd":1,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.87}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.71699999999999997}]},{"osd":3,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.036},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.99399999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.1220000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.018}]},{"osd":7,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.90000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4808704,"data_stored":40588859,"data_compressed":832024,"data_compressed_allocated":3108864,"data_compressed_original":39543368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5787648,"data_stored":54052990,"data_compressed":922546,"data_compressed_allocated":4313088,"data_compressed_original":53198173,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3792896,"data_stored":35736716,"data_compressed":558602,"data_compressed_allocated":2326528,"data_compressed_original":34871650,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2277376,"data_stored":11875747,"data_compressed":563604,"data_compressed_allocated":1290240,"data_compressed_original":11292164,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3629056,"data_stored":32810962,"data_compressed":543265,"data_compressed_allocated":2355200,"data_compressed_original":32060315,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4513792,"data_stored":45513286,"data_compressed":764306,"data_compressed_allocated":3063808,"data_compressed_original":44638075,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4956160,"data_stored":52424607,"data_compressed":849341,"data_compressed_allocated":3567616,"data_compressed_original":51630770,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2256896,"data_stored":19202459,"data_compressed":357688,"data_compressed_allocated":1261568,"data_compressed_original":18593858,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:08:15.203 INFO:tasks.thrashosds.thrasher:Setting 2.7 to [1, 4, 0, 3] 2024-09-16T14:08:15.203 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.7', '1', '4', '0', '3'] 2024-09-16T14:08:15.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.7 1 4 0 3 2024-09-16T14:08:15.251 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:15.251 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:15.261 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:15.307 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:15.302+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:15.362 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:15.362 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:15.372 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:15.379 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:15.374+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:15.417 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone27 from 0xd2b7742 (overlap 0x449990d) to 0x36e140a 2024-09-16T14:08:15.480 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:15.481 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:15.481 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:15.477+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:15.489 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:15.582 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:15.574+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:15.582 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:15.582 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:15.591 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:15.685 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:15.678+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:15.692 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:15.693 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:15.703 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:15.786 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:15.778+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:15.799 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:15.799 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:15.808 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:15.889 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:15.885+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:15.906 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:15.907 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:15.916 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:15.990 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:15.982+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:16.016 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:16.016 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:16.025 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:16.092 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:16.089+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:16.111 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.7 pg_upmap_items mapping to [1->4,0->3] 2024-09-16T14:08:16.114 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:08:16.127 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:16.127 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:16.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:16.192 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:16.186+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:16.281 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:16.281 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:16.289 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-16T14:08:16.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:16.292 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:16.286+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:16.319 INFO:teuthology.orchestra.run.smithi079.stdout:3034 write 0xc541380 thru 0xc54cfdf (0xbc60 bytes) 2024-09-16T14:08:16.333 INFO:teuthology.orchestra.run.smithi079.stdout:3036 punch from 0xac616b6 to 0xac68ecb, (0x7815 bytes) 2024-09-16T14:08:16.333 INFO:teuthology.orchestra.run.smithi079.stdout:3037 write 0xb1464cb thru 0xb156400 (0xff36 bytes) 2024-09-16T14:08:16.342 INFO:teuthology.orchestra.run.smithi079.stdout:3038 read 0xc10baa6 thru 0xc10fb87 (0x40e2 bytes) 2024-09-16T14:08:16.345 INFO:teuthology.orchestra.run.smithi079.stdout:3039 trunc from 0xd2b7742 to 0xbea14a0 2024-09-16T14:08:16.377 INFO:teuthology.orchestra.run.smithi079.stdout:3040 read 0x4bc861a thru 0x4bcb6e2 (0x30c9 bytes) 2024-09-16T14:08:16.378 INFO:teuthology.orchestra.run.smithi079.stdout:3041 punch from 0x40506d7 to 0x405d7a3, (0xd0cc bytes) 2024-09-16T14:08:16.393 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:16.386+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:16.393 INFO:teuthology.orchestra.run.smithi079.stdout:3042 trunc from 0xbea14a0 to 0x2a56356 2024-09-16T14:08:16.394 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:16.394 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:16.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:16.449 INFO:teuthology.orchestra.run.smithi079.stdout:3043 writesame 0x5a3b2be thru 0x5a3dd83 data_size 0x32(0x2ac6 bytes) 2024-09-16T14:08:16.449 INFO:teuthology.orchestra.run.smithi079.stdout:3044 write 0x13d7a6b thru 0x13e1bf4 (0xa18a bytes) 2024-09-16T14:08:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:3046 punch from 0x3d1450 to 0x3dcc9f, (0xb84f bytes) 2024-09-16T14:08:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:3047 read 0x51e83e7 thru 0x51eb7a8 (0x33c2 bytes) 2024-09-16T14:08:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:3048 read 0x5456ad1 thru 0x545801a (0x154a bytes) 2024-09-16T14:08:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:3050 read 0x4a11289 thru 0x4a186d2 (0x744a bytes) 2024-09-16T14:08:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:3053 read 0x4ee333e thru 0x4ef130a (0xdfcd bytes) 2024-09-16T14:08:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:3055 read 0x57fddfd thru 0x5806689 (0x888d bytes) 2024-09-16T14:08:16.468 INFO:teuthology.orchestra.run.smithi079.stdout:3056 write 0x27ee14 thru 0x281d1f (0x2f0c bytes) 2024-09-16T14:08:16.486 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:16.486 INFO:teuthology.orchestra.run.smithi079.stdout:3058 trunc from 0x5a3dd84 to 0x5a90a0b 2024-09-16T14:08:16.494 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:16.486+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:16.496 INFO:teuthology.orchestra.run.smithi079.stdout:3059 writesame 0x557f2d0 thru 0x558d02e data_size 0x1(0xdd5f bytes) 2024-09-16T14:08:16.512 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:16.512 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:16.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:16.555 INFO:teuthology.orchestra.run.smithi079.stdout:3060 writesame 0xec26e3c thru 0xec2afff data_size 0xc(0x41c4 bytes) 2024-09-16T14:08:16.556 INFO:teuthology.orchestra.run.smithi079.stdout:3061 write 0x996edf6 thru 0x9975223 (0x642e bytes) 2024-09-16T14:08:16.566 INFO:teuthology.orchestra.run.smithi079.stdout:3065 read 0xb35cccb thru 0xb35d645 (0x97b bytes) 2024-09-16T14:08:16.566 INFO:teuthology.orchestra.run.smithi079.stdout:3066 punch from 0xbee363e to 0xbeeb47a, (0x7e3c bytes) 2024-09-16T14:08:16.566 INFO:teuthology.orchestra.run.smithi079.stdout:3067 writesame 0x432c40c thru 0x4331f35 data_size 0x7(0x5b2a bytes) 2024-09-16T14:08:16.575 INFO:teuthology.orchestra.run.smithi079.stdout:3068 punch from 0x7aa5793 to 0x7ab1077, (0xb8e4 bytes) 2024-09-16T14:08:16.575 INFO:teuthology.orchestra.run.smithi079.stdout:3069 punch from 0x22963b5 to 0x229ff6d, (0x9bb8 bytes) 2024-09-16T14:08:16.575 INFO:teuthology.orchestra.run.smithi079.stdout:3070 punch from 0x542f008 to 0x5437534, (0x852c bytes) 2024-09-16T14:08:16.575 INFO:teuthology.orchestra.run.smithi079.stdout:3071 writesame 0x8740c8a thru 0x8748203 data_size 0xb(0x757a bytes) 2024-09-16T14:08:16.583 INFO:teuthology.orchestra.run.smithi079.stdout:3073 write 0xe87caea thru 0xe886ec1 (0xa3d8 bytes) 2024-09-16T14:08:16.595 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:16.590+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:16.604 INFO:teuthology.orchestra.run.smithi079.stdout:3074 trunc from 0xec2b000 to 0x7c83d62 2024-09-16T14:08:16.618 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:16.618 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:16.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:16.638 INFO:teuthology.orchestra.run.smithi079.stdout:3075 trunc from 0x7c83d62 to 0xe00dd4d 2024-09-16T14:08:16.648 INFO:teuthology.orchestra.run.smithi079.stdout:3077 write 0x3bf44f0 thru 0x3bfed2d (0xa83e bytes) 2024-09-16T14:08:16.673 INFO:teuthology.orchestra.run.smithi079.stdout:3078 trunc from 0xe00dd4d to 0x444bca4 2024-09-16T14:08:16.694 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:16.695 INFO:teuthology.orchestra.run.smithi079.stdout:3082 punch from 0x43610e5 to 0x43629c7, (0x18e2 bytes) 2024-09-16T14:08:16.695 INFO:teuthology.orchestra.run.smithi079.stdout:3083 writesame 0x1b01ecf thru 0x1b09c48 data_size 0x1(0x7d7a bytes) 2024-09-16T14:08:16.695 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:16.690+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:16.738 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:16.738 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:16.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:16.756 INFO:teuthology.orchestra.run.smithi079.stdout:3084 write 0xa024acf thru 0xa0306bd (0xbbef bytes) 2024-09-16T14:08:16.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:16.767 INFO:teuthology.orchestra.run.smithi079.stdout:3086 trunc from 0xa0306be to 0x8d2fb1a 2024-09-16T14:08:16.785 INFO:teuthology.orchestra.run.smithi079.stdout:3089 read 0x472dd thru 0x4ecbd (0x79e1 bytes) 2024-09-16T14:08:16.796 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:16.790+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:16.805 INFO:teuthology.orchestra.run.smithi079.stdout:3090 write 0xe87cdbe thru 0xe885bc9 (0x8e0c bytes) 2024-09-16T14:08:16.814 INFO:teuthology.orchestra.run.smithi079.stdout:3091 punch from 0xabf702e to 0xabffebb, (0x8e8d bytes) 2024-09-16T14:08:16.815 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:16.815 INFO:teuthology.orchestra.run.smithi079.stdout:3093 flatten 2024-09-16T14:08:16.888 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:16.888 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:16.897 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:16.890+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:16.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:16.907 INFO:teuthology.orchestra.run.smithi079.stdout:3094 read 0x91ed9d1 thru 0x91f1687 (0x3cb7 bytes) 2024-09-16T14:08:16.907 INFO:teuthology.orchestra.run.smithi079.stdout:3095 read 0x4076f92 thru 0x4082317 (0xb386 bytes) 2024-09-16T14:08:16.909 INFO:teuthology.orchestra.run.smithi079.stdout:3096 writesame 0x100ddad thru 0x1013cfe data_size 0x62(0x5f52 bytes) 2024-09-16T14:08:16.913 INFO:teuthology.orchestra.run.smithi079.stdout:3097 writesame 0x2351b5b thru 0x235d6d8 data_size 0x67(0xbb7e bytes) 2024-09-16T14:08:16.916 INFO:teuthology.orchestra.run.smithi079.stdout:3099 read 0xe37e71c thru 0xe38249e (0x3d83 bytes) 2024-09-16T14:08:16.916 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:16.917 INFO:teuthology.orchestra.run.smithi079.stdout:3101 punch from 0xcde10a6 to 0xcdeb03f, (0x9f99 bytes) 2024-09-16T14:08:16.917 INFO:teuthology.orchestra.run.smithi079.stdout:3102 punch from 0xd4e7a54 to 0xd4ef3c7, (0x7973 bytes) 2024-09-16T14:08:16.917 INFO:teuthology.orchestra.run.smithi079.stdout:3103 punch from 0x895071e to 0x8960054, (0xf936 bytes) 2024-09-16T14:08:16.917 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:16.917 INFO:teuthology.orchestra.run.smithi079.stdout:3106 writesame 0xc05516d thru 0xc0557c1 data_size 0x1(0x655 bytes) 2024-09-16T14:08:16.921 INFO:teuthology.orchestra.run.smithi079.stdout:3107 write 0xd24bcae thru 0xd2514ee (0x5841 bytes) 2024-09-16T14:08:16.929 INFO:teuthology.orchestra.run.smithi079.stdout:3109 read 0x844aa5e thru 0x845aa1f (0xffc2 bytes) 2024-09-16T14:08:16.929 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:16.929 INFO:teuthology.orchestra.run.smithi079.stdout:3111 writesame 0xe800a2b thru 0xe805431 data_size 0x1(0x4a07 bytes) 2024-09-16T14:08:16.934 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:16.945 INFO:teuthology.orchestra.run.smithi079.stdout:3112 write 0x2860d2a thru 0x28703ec (0xf6c3 bytes) 2024-09-16T14:08:16.949 INFO:teuthology.orchestra.run.smithi079.stdout:3114 punch from 0xc5a4275 to 0xc5af917, (0xb6a2 bytes) 2024-09-16T14:08:16.949 INFO:teuthology.orchestra.run.smithi079.stdout:3115 write 0x8209704 thru 0x8211f01 (0x87fe bytes) 2024-09-16T14:08:16.951 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:16.956 INFO:teuthology.orchestra.run.smithi079.stdout:3116 write 0x2e573cf thru 0x2e60840 (0x9472 bytes) 2024-09-16T14:08:16.963 INFO:teuthology.orchestra.run.smithi079.stdout:3117 read 0xe41b3d6 thru 0xe420b6f (0x579a bytes) 2024-09-16T14:08:16.963 INFO:teuthology.orchestra.run.smithi079.stdout:3118 punch from 0xe2e6867 to 0xe2ec7f8, (0x5f91 bytes) 2024-09-16T14:08:16.973 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:16.986 INFO:teuthology.orchestra.run.smithi079.stdout:3120 trunc from 0xe885bca to 0x960f2f 2024-09-16T14:08:16.994 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:16.998 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:16.990+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:17.003 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:17.004 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:17.004 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:17.004 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:17.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:17.020 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:17.035 INFO:teuthology.orchestra.run.smithi079.stdout:3121 write 0x7ed9979 thru 0x7ee0709 (0x6d91 bytes) 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:17.041 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:17.044 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:17.044 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:17.045 INFO:teuthology.orchestra.run.smithi079.stdout:3122 trunc from 0x7ee070a to 0x92ac7ec 2024-09-16T14:08:17.055 INFO:teuthology.orchestra.run.smithi079.stdout:3123 write 0x7429976 thru 0x742d961 (0x3fec bytes) 2024-09-16T14:08:17.064 INFO:teuthology.orchestra.run.smithi079.stdout:3124 trunc from 0x92ac7ec to 0xb407349 2024-09-16T14:08:17.093 INFO:teuthology.orchestra.run.smithi079.stdout:3125 trunc from 0xb407349 to 0x1056759 2024-09-16T14:08:17.098 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:17.090+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:17.126 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:17.126 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:17.126 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:17.126 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:17.127 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:17.129 INFO:teuthology.orchestra.run.smithi079.stdout:3126 write 0x40e5648 thru 0x40efaa5 (0xa45e bytes) 2024-09-16T14:08:17.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:17.136 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:17.136 INFO:teuthology.orchestra.run.smithi079.stdout:3128 read 0x37a8c9a thru 0x37b67aa (0xdb11 bytes) 2024-09-16T14:08:17.147 INFO:teuthology.orchestra.run.smithi079.stdout:3129 writesame 0x49d1cba thru 0x49db4ac data_size 0x1(0x97f3 bytes) 2024-09-16T14:08:17.147 INFO:teuthology.orchestra.run.smithi079.stdout:3130 read 0x2609009 thru 0x2618bfb (0xfbf3 bytes) 2024-09-16T14:08:17.148 INFO:teuthology.orchestra.run.smithi079.stdout:3131 read 0x129f9b0 thru 0x12a7dc5 (0x8416 bytes) 2024-09-16T14:08:17.148 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:17.148 INFO:teuthology.orchestra.run.smithi079.stdout:3135 punch from 0x1d21591 to 0x1d22ff4, (0x1a63 bytes) 2024-09-16T14:08:17.148 INFO:teuthology.orchestra.run.smithi079.stdout:3136 read 0x1fc72c3 thru 0x1fd0fc5 (0x9d03 bytes) 2024-09-16T14:08:17.148 INFO:teuthology.orchestra.run.smithi079.stdout:3137 read 0x471c372 thru 0x4729985 (0xd614 bytes) 2024-09-16T14:08:17.148 INFO:teuthology.orchestra.run.smithi079.stdout:3138 read 0x48e2c49 thru 0x48e7265 (0x461d bytes) 2024-09-16T14:08:17.175 INFO:teuthology.orchestra.run.smithi079.stdout:3139 write 0xde6b8eb thru 0xde78c94 (0xd3aa bytes) 2024-09-16T14:08:17.187 INFO:teuthology.orchestra.run.smithi079.stdout:3140 punch from 0xb57a3f2 to 0xb58087c, (0x648a bytes) 2024-09-16T14:08:17.187 INFO:teuthology.orchestra.run.smithi079.stdout:3142 writesame 0x1c1fcaf thru 0x1c2df38 data_size 0x6b(0xe28a bytes) 2024-09-16T14:08:17.198 INFO:teuthology.orchestra.run.smithi079.stdout:3143 read 0xf694e0 thru 0xf6dd27 (0x4848 bytes) 2024-09-16T14:08:17.198 INFO:teuthology.orchestra.run.smithi079.stdout:3144 write 0x27c8dce thru 0x27d24bd (0x96f0 bytes) 2024-09-16T14:08:17.199 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:17.194+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:17.205 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:17.205 INFO:teuthology.orchestra.run.smithi079.stdout:3146 write 0xbe83bb8 thru 0xbe87785 (0x3bce bytes) 2024-09-16T14:08:17.211 INFO:teuthology.orchestra.run.smithi079.stdout:3147 write 0x501ba9c thru 0x502a43c (0xe9a1 bytes) 2024-09-16T14:08:17.221 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:17.221 INFO:teuthology.orchestra.run.smithi079.stdout:3149 write 0x22512dd thru 0x2254aea (0x380e bytes) 2024-09-16T14:08:17.227 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:17.227 INFO:teuthology.orchestra.run.smithi079.stdout:3151 clone 29 order 20 su 4096 sc 6 2024-09-16T14:08:17.245 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86627 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 115712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.070213953, 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057990535000000003, 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.247 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86627 2024-09-16T14:08:17.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.247 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:17.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.79e-07 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172073+0000", 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.53e-06 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172077+0000", 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172079+0000", 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:17.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224325+0000", 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052246133 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224394+0000", 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8960000000000004e-05 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.230062+0000", 2024-09-16T14:08:17.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056682820000000002 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86989 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.048605107, 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058737710999999998, 2024-09-16T14:08:17.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86989 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193681+0000", 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193682+0000", 2024-09-16T14:08:17.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6169999999999999e-06 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193687+0000", 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7210000000000002e-06 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193690+0000", 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.072e-06 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.246881+0000", 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.053191117000000003 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252309+0000", 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054275979999999996 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252418+0000", 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109255 2024-09-16T14:08:17.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86995 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.048578904999999, 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059640662999999997, 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86995 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197498+0000", 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037915219999999999 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197504+0000", 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7520000000000002e-06 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197515+0000", 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.117e-05 2024-09-16T14:08:17.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197521+0000", 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.083e-06 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253121+0000", 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055600254000000002 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253227+0000", 2024-09-16T14:08:17.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010607000000000001 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253347+0000", 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119812 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87017 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:17.260 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044643169, 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676182999999998, 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87017 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:17.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197643+0000", 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197645+0000", 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197652+0000", 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8430000000000004e-06 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197656+0000", 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:17.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254046+0000", 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056390033999999999 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254205+0000", 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158639 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254318+0000", 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113916 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87019 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044625592999999, 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056580459, 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87019 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:17.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197663+0000", 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.396e-06 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197669+0000", 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4099999999999996e-06 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197673+0000", 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254066+0000", 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056392870999999997 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254160+0000", 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4574000000000006e-05 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254240+0000", 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0125999999999998e-05 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.269 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87023 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 193536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044608726, 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056736048999999997, 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87023 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197679+0000", 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197686+0000", 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197690+0000", 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.177e-06 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254263+0000", 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056573364000000001 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254334+0000", 2024-09-16T14:08:17.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0574000000000006e-05 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254413+0000", 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079999999999995e-05 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87027 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:17.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044591603000001, 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056974745, 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87027 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:17.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2599999999999998e-07 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197696+0000", 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1679999999999998e-06 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197703+0000", 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4389999999999997e-06 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197707+0000", 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254383+0000", 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676509999999999 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254483+0000", 2024-09-16T14:08:17.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9692999999999996e-05 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254669+0000", 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018580300000000001 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87029 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 197632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044574766, 2024-09-16T14:08:17.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057332283999999997, 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87029 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197713+0000", 2024-09-16T14:08:17.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.199e-06 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197719+0000", 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.699e-06 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197723+0000", 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000002e-06 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254709+0000", 2024-09-16T14:08:17.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056986536999999997 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254790+0000", 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0192999999999996e-05 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255043+0000", 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025339500000000001 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87031 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 234496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044558473999999, 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057120535, 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87031 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.284 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0600000000000001e-07 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197729+0000", 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0499999999999999e-06 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197736+0000", 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2249999999999997e-06 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197739+0000", 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6839999999999998e-06 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254697+0000", 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056957902999999997 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254776+0000", 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-16T14:08:17.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254848+0000", 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1044999999999995e-05 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87035 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 198656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.04454247, 2024-09-16T14:08:17.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057340290000000002, 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87035 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197744+0000", 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6499999999999999e-07 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197746+0000", 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0169999999999999e-06 2024-09-16T14:08:17.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197752+0000", 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1040000000000003e-06 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197755+0000", 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.782e-06 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254869+0000", 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057113766000000003 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254971+0000", 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101932 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255083+0000", 2024-09-16T14:08:17.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112124 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87039 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 238592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044526059999999, 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057586228000000003, 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87039 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:17.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197760+0000", 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197762+0000", 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.829e-06 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197768+0000", 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5470000000000002e-06 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197772+0000", 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4699999999999998e-06 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255109+0000", 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057337232000000002 2024-09-16T14:08:17.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255184+0000", 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5593000000000006e-05 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255346+0000", 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016120899999999999 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87041 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 202752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044510005999999, 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057827176000000001, 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87041 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197776+0000", 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197778+0000", 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3709999999999998e-06 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197784+0000", 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.054e-06 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197787+0000", 2024-09-16T14:08:17.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1879999999999998e-06 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255397+0000", 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057609059999999997 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255483+0000", 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6415999999999994e-05 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255603+0000", 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119703 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87043 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 203776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044493839000001, 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057723202000000001, 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87043 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:17.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197801+0000", 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5869999999999993e-06 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197807+0000", 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197811+0000", 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0099999999999997e-06 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255377+0000", 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057566063000000001 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255450+0000", 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3192e-05 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255515+0000", 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4543999999999997e-05 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87045 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044469646, 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057864405000000001, 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87045 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-16T14:08:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197819+0000", 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.311e-06 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197824+0000", 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6980000000000003e-06 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197828+0000", 2024-09-16T14:08:17.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2339999999999999e-06 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255537+0000", 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057709979000000002 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255613+0000", 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5807000000000001e-05 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255680+0000", 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6935999999999997e-05 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87051 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.044454494, 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058010645999999999, 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87051 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200527+0000", 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026964990000000002 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200532+0000", 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8899999999999998e-06 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200541+0000", 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1249999999999993e-06 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200546+0000", 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4129999999999997e-06 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255630+0000", 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055083612999999997 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255711+0000", 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1934000000000005e-05 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255842+0000", 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013017200000000001 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87559 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 100352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.041620137999999, 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056509688000000002, 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87559 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200666+0000", 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200668+0000", 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.21e-06 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200674+0000", 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.896e-06 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200678+0000", 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7050000000000001e-06 2024-09-16T14:08:17.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.256898+0000", 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056220883999999999 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257014+0000", 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011530499999999999 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257175+0000", 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016127200000000001 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87568 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.041588159, 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056739408999999998, 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87568 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200698+0000", 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200700+0000", 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.272e-06 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200706+0000", 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7080000000000002e-06 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200709+0000", 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.394e-06 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257221+0000", 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056512383999999999 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257319+0000", 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7510000000000007e-05 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257437+0000", 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011773399999999999 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.324 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87570 3.e 3:73b2712a:::rbd_data.113619b3ee73.0000000000000027:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.041572827, 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056717201000000002, 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87570 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201471+0000", 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075828899999999999 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201480+0000", 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.448e-06 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201494+0000", 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3162999999999999e-05 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201500+0000", 2024-09-16T14:08:17.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0090000000000004e-06 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257205+0000", 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055705603999999999 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257330+0000", 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124972 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257430+0000", 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9716000000000005e-05 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87651 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 136192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.036673121, 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056431633000000002, 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87651 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205613+0000", 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999998e-07 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205614+0000", 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:17.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205619+0000", 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.758e-06 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205622+0000", 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0690000000000002e-06 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.261887+0000", 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056265241000000001 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.261965+0000", 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7671999999999998e-05 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262044+0000", 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9320999999999995e-05 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87773 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 63488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.035795813, 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056458201, 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87773 2024-09-16T14:08:17.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:17.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207001+0000", 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051104000000000004 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207006+0000", 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207019+0000", 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2266000000000001e-05 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207025+0000", 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2029999999999997e-06 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262711+0000", 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055686539 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262801+0000", 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9895999999999994e-05 2024-09-16T14:08:17.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262948+0000", 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146642 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:17.338 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:17.340 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:17.340 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:17.334+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:17.441 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:17.434+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:17.447 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:17.447 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:17.448 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:17.448 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:17.457 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:17.541 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:17.534+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:17.556 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:17.557 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:17.557 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:17.557 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:17.557 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:17.566 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:17.642 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:17.634+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:17.670 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1143 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.328793843, 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018939362000000001, 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1143 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336881+0000", 2024-09-16T14:08:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.44e-07 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336884+0000", 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8159999999999998e-06 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336891+0000", 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336895+0000", 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0820000000000001e-06 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349584+0000", 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012689283000000001 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.349708+0000", 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123793 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350151+0000", 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044259900000000001 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352993+0000", 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028426979999999998 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355720+0000", 2024-09-16T14:08:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002726642 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355762+0000", 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2116000000000001e-05 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355820+0000", 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8118999999999997e-05 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.328775737000001, 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.328753339999999, 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.328146629000003, 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.328072302000002, 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.357147509999997, 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.729188929000003, 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.199028462999998, 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.551479931999999, 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101781 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005234139000001, 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019058275, 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101781 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:08:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660443+0000", 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660450+0000", 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999997e-06 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660454+0000", 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0029999999999996e-06 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679188+0000", 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018734869000000001 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679271+0000", 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2721999999999998e-05 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679499+0000", 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227612 2024-09-16T14:08:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101783 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005216961, 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019291800000000001, 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101783 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4499999999999997e-07 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660465+0000", 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4659999999999996e-06 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660471+0000", 2024-09-16T14:08:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5250000000000002e-06 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660480+0000", 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5779999999999999e-06 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679530+0000", 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019050085000000001 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679633+0000", 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103341 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679750+0000", 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011636 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005189692, 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005172501000001, 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005135484, 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005114808, 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005072628000001, 2024-09-16T14:08:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:17.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005038403, 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:17.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.763 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.005008687, 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:17.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:17.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:17.768 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.004989025, 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:17.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.9962289, 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:17.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:17.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:17.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:17.777 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:17.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:17.779 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:17.777+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:17.880 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:17.877+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:17.886 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:17.886 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:17.886 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:17.886 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:17.895 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:17.981 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:17.977+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:17.991 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:17.991 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:17.991 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:17.992 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:17.992 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:18.001 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:18.081 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:18.074+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:18.103 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:18.103 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102411 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.428702585, 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050639235999999997, 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102411 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670933+0000", 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0499999999999999e-07 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670939+0000", 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8409999999999996e-06 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670945+0000", 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8599999999999998e-06 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670949+0000", 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.653e-06 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721301+0000", 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050352791000000001 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721435+0000", 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013400799999999999 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721572+0000", 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013667799999999999 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102417 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 156672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.428682693000001, 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050881124999999999, 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102417 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670953+0000", 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670954+0000", 2024-09-16T14:08:18.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670956+0000", 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.215e-06 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670962+0000", 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8320000000000002e-06 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.670967+0000", 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5539999999999997e-06 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721604+0000", 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050636674 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721699+0000", 2024-09-16T14:08:18.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5441999999999999e-05 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721834+0000", 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134704 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102420 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 3072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:18.113 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.42861115, 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051052476999999999, 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102420 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:18.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671024+0000", 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671025+0000", 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.018e-06 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671030+0000", 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8189999999999999e-06 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671042+0000", 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1902000000000001e-05 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.671048+0000", 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2149999999999998e-06 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721866+0000", 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050817648 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.721964+0000", 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7746999999999998e-05 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.722077+0000", 2024-09-16T14:08:18.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011312800000000001 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103091 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.40623553, 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050794981000000003, 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103091 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:18.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693400+0000", 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7800000000000002e-07 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693403+0000", 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2340000000000001e-06 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693408+0000", 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.806e-06 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693412+0000", 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5939999999999998e-06 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744002+0000", 2024-09-16T14:08:18.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050590334000000001 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744093+0000", 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0394000000000002e-05 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744195+0000", 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102241 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.122 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103097 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.406219362, 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050990884, 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103097 2024-09-16T14:08:18.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693416+0000", 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693839+0000", 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00042325899999999999 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693844+0000", 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9640000000000002e-06 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693855+0000", 2024-09-16T14:08:18.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0951999999999999e-05 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693863+0000", 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2309999999999999e-06 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744227+0000", 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050364642000000001 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744304+0000", 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7088000000000005e-05 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744407+0000", 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102748 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103100 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 204800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.127 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.405766362, 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050768523000000003, 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103100 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693869+0000", 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693870+0000", 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693889+0000", 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9040999999999999e-05 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693902+0000", 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3298e-05 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693907+0000", 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9690000000000002e-06 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744435+0000", 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050527739000000002 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744523+0000", 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8225000000000003e-05 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744638+0000", 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011479700000000001 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103114 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 56320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.40572294, 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050930699000000003, 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103114 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693913+0000", 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3300000000000001e-07 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693916+0000", 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4540000000000001e-06 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693922+0000", 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1199999999999999e-06 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693927+0000", 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9089999999999999e-06 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744661+0000", 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050733893000000002 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744735+0000", 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4637999999999999e-05 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744843+0000", 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107952 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103119 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 133120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.405704659, 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051196642000000001, 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103119 2024-09-16T14:08:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693931+0000", 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693933+0000", 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0540000000000002e-06 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693939+0000", 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2090000000000002e-06 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693944+0000", 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.744904+0000", 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050960022000000001 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745010+0000", 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106399 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745127+0000", 2024-09-16T14:08:18.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117451 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103122 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 168960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.40568773, 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051372329000000001, 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103122 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693948+0000", 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6199999999999998e-07 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693951+0000", 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3029999999999998e-06 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693956+0000", 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6829999999999996e-06 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693960+0000", 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6660000000000001e-06 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745168+0000", 2024-09-16T14:08:18.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051207616999999997 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745244+0000", 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6412e-05 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745320+0000", 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6186000000000005e-05 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103127 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.405671712, 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051736246, 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103127 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693964+0000", 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693966+0000", 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2120000000000002e-06 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693972+0000", 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8680000000000004e-06 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693976+0000", 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4850000000000001e-06 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745485+0000", 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051508784000000002 2024-09-16T14:08:18.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745581+0000", 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6396000000000003e-05 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745700+0000", 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011909 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103135 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 20480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.405656041, 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051887555000000002, 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103135 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.154 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693979+0000", 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693980+0000", 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693982+0000", 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8300000000000001e-06 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693989+0000", 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.691e-06 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693992+0000", 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3699999999999999e-06 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745733+0000", 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051741405999999997 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745786+0000", 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2967000000000003e-05 2024-09-16T14:08:18.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745867+0000", 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0668999999999996e-05 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103142 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.405639812, 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.052104512999999998, 2024-09-16T14:08:18.158 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103142 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.693996+0000", 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697955+0000", 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039591230000000002 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697960+0000", 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000001e-06 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697971+0000", 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1094000000000001e-05 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.697977+0000", 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7989999999999999e-06 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745893+0000", 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.047915755999999997 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.745972+0000", 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9406999999999994e-05 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.746100+0000", 2024-09-16T14:08:18.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000127961 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103237 3.1f 3:faf97aec:::rbd_data.113619b3ee73.0000000000000005:16 [read 12288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.40131689, 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050774314000000001, 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.163 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103237 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.698319+0000", 2024-09-16T14:08:18.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701919+0000", 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036008490000000002 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701930+0000", 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0882e-05 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701940+0000", 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9620000000000005e-06 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.701953+0000", 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945999999999999e-05 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.748858+0000", 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.046904281999999999 2024-09-16T14:08:18.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.748955+0000", 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7583000000000004e-05 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.749093+0000", 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013781000000000001 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103351 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 184320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.397303033, 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.05063119, 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103351 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702332+0000", 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702333+0000", 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.72e-07 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702335+0000", 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8220000000000001e-06 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702340+0000", 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0579999999999997e-06 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702344+0000", 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2479999999999998e-06 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752784+0000", 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050439954000000002 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752882+0000", 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T14:08:18.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752964+0000", 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1549000000000001e-05 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:103357 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.397287539000001, 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050776609, 2024-09-16T14:08:18.172 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103357 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.702348+0000", 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705848+0000", 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035001839999999999 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705854+0000", 2024-09-16T14:08:18.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4779999999999999e-06 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705863+0000", 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5529999999999998e-06 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.705869+0000", 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1800000000000001e-06 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.752985+0000", 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.04711605 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.753035+0000", 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9997000000000003e-05 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.753125+0000", 2024-09-16T14:08:18.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9166999999999995e-05 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104426 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 152576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.363341376999999, 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050895079000000003, 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104426 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736294+0000", 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5999999999999999e-07 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736296+0000", 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9539999999999998e-06 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736302+0000", 2024-09-16T14:08:18.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1599999999999997e-06 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736305+0000", 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1609999999999999e-06 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787004+0000", 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050698852000000003 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787087+0000", 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3418999999999995e-05 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787189+0000", 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102173 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104434 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 229376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.363326482, 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051134456000000002, 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104434 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736309+0000", 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736311+0000", 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736317+0000", 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3709999999999999e-06 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736320+0000", 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4230000000000001e-06 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787233+0000", 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050912675999999997 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787329+0000", 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6628999999999999e-05 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787443+0000", 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011399400000000001 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.186 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104446 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 4096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.362986018000001, 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050967213999999997, 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104446 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736649+0000", 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736650+0000", 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6700000000000002e-07 2024-09-16T14:08:18.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736655+0000", 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6940000000000003e-06 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736665+0000", 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0121e-05 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736671+0000", 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4609999999999999e-06 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787463+0000", 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050792342999999997 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787528+0000", 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4453000000000001e-05 2024-09-16T14:08:18.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787617+0000", 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9174999999999999e-05 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104454 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.362949393999999, 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051215203000000001, 2024-09-16T14:08:18.191 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104454 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736686+0000", 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736687+0000", 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0599999999999999e-07 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736690+0000", 2024-09-16T14:08:18.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736697+0000", 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1130000000000003e-06 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.736701+0000", 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5419999999999999e-06 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787711+0000", 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.051009546000000003 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787799+0000", 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7961999999999996e-05 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.787901+0000", 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102416 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.195 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:104465 3.1f 3:feef7286:::rbd_data.113619b3ee73.0000000000000007:16 [read 193536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.361948788999999, 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050750456999999999, 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.196 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 104465 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737687+0000", 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6300000000000005e-07 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737690+0000", 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7310000000000002e-06 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737696+0000", 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1530000000000002e-06 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.737700+0000", 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8909999999999997e-06 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788242+0000", 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.050542301999999997 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788329+0000", 2024-09-16T14:08:18.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6527999999999996e-05 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.788437+0000", 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108189 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.200 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:18.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:08:18.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:18.203 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:18.201+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:18.303 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:18.301+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:18.361 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:18.361 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:18.361 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:18.371 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:18.403 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:18.398+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:18.475 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:18.475 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:18.475 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:18.475 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:18.475 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:18.484 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:18.527 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:18.522+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:18.604 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:18.598+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:18.633 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:18.633 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.254766277, 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.634 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:08:18.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.638 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.250834858999999, 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:18.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:08:18.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.250810667, 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:08:18.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.247091396, 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.648 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:08:18.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.243174631, 2024-09-16T14:08:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:08:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.238792352000001, 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.657 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:08:18.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.238768511, 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:08:18.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:08:18.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:18.666 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.238753322999999, 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:18.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:18.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:08:18.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.238738737, 2024-09-16T14:08:18.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:08:18.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.238635087, 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.676 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.238607361, 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.235933745000001, 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.235907031, 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.235866864, 2024-09-16T14:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.235777149, 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.235368344999999, 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.235268161, 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.234369523, 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.227170791000001, 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.225002499, 2024-09-16T14:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:18.729 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-16T14:08:18.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:18.731 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:18.729+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:18.831 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:18.826+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:18.861 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:18.861 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:18.861 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:18.861 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:18.872 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:18.932 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:18.926+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:18.986 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:18.987 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:18.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:19.033 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:19.026+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:19.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:19.112 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.699055666, 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:08:19.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:08:19.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.861093343, 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.120 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:08:19.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:08:19.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33385435499999999, 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:19.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:08:19.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33383112399999998, 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.131 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:08:19.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33380817699999998, 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:08:19.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:08:19.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.140 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33379166799999999, 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:08:19.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33377404199999999, 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.32785829799999999, 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:08:19.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:08:19.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:19.154 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.32784089300000002, 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:19.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:08:19.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.32748652700000003, 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:08:19.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:08:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:08:19.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:08:19.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.165 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.324342668, 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:08:19.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:08:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:08:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:08:19.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.324327382, 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.171 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:19.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:08:19.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.324310973, 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:08:19.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:08:19.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.32423577599999998, 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:08:19.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:19.185 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.324138229, 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:08:19.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.32411298799999999, 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.190 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.194 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.32408393400000002, 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:08:19.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:08:19.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.320819945, 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.199 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:08:19.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.320801744, 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:08:19.203 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:08:19.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.207 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.32076256600000003, 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:19.208 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:08:19.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:19.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:19.212 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:19.214 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:08:19.215 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:19.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:19.215 INFO:teuthology.orchestra.run.smithi079.stdout:3153 trunc from 0xde78c95 to 0x1915e26 2024-09-16T14:08:19.216 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:19.216 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:19.213+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:19.234 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:19.258 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:19.281 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:19.284 INFO:teuthology.orchestra.run.smithi079.stdout:3154 write 0x9913016 thru 0x991a61c (0x7607 bytes) 2024-09-16T14:08:19.297 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:19.306 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:19.311 INFO:teuthology.orchestra.run.smithi079.stdout:3156 trunc from 0x991a61d to 0x111fdb5 2024-09-16T14:08:19.314 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:19.306+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:19.327 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:19.327 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:19.327 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:19.327 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:19.328 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:19.328 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:19.328 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:19.328 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:19.328 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:19.330 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:19.331 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:19.332 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:19.332 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:19.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:19.363 INFO:teuthology.orchestra.run.smithi079.stdout:3157 trunc from 0x111fdb5 to 0xcb4bb1d 2024-09-16T14:08:19.374 INFO:teuthology.orchestra.run.smithi079.stdout:3160 punch from 0xbc8bfcc to 0xbc9b859, (0xf88d bytes) 2024-09-16T14:08:19.376 INFO:teuthology.orchestra.run.smithi079.stdout:3161 trunc from 0xcb4bb1d to 0xdb12445 2024-09-16T14:08:19.388 INFO:teuthology.orchestra.run.smithi079.stdout:3162 writesame 0x502d74f thru 0x5031ab1 data_size 0x1(0x4363 bytes) 2024-09-16T14:08:19.406 INFO:teuthology.orchestra.run.smithi079.stdout:3163 write 0x5997dc8 thru 0x59a5037 (0xd270 bytes) 2024-09-16T14:08:19.415 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:19.410+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:19.418 INFO:teuthology.orchestra.run.smithi079.stdout:3164 read 0x7865700 thru 0x78704ce (0xadcf bytes) 2024-09-16T14:08:19.419 INFO:teuthology.orchestra.run.smithi079.stdout:3165 read 0x53b6925 thru 0x53b7759 (0xe35 bytes) 2024-09-16T14:08:19.419 INFO:teuthology.orchestra.run.smithi079.stdout:3166 read 0x1a3034b thru 0x1a37dfc (0x7ab2 bytes) 2024-09-16T14:08:19.419 INFO:teuthology.orchestra.run.smithi079.stdout:3167 writesame 0x38d0fb5 thru 0x38d74ce data_size 0x1(0x651a bytes) 2024-09-16T14:08:19.448 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:19.448 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:19.449 INFO:teuthology.orchestra.run.smithi079.stdout:3168 trunc from 0xdb12445 to 0x52807cf 2024-09-16T14:08:19.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:19.472 INFO:teuthology.orchestra.run.smithi079.stdout:3169 writesame 0x46a9e02 thru 0x46b16a5 data_size 0xe(0x78a4 bytes) 2024-09-16T14:08:19.490 INFO:teuthology.orchestra.run.smithi079.stdout:3170 write 0x46bb3fa thru 0x46c2329 (0x6f30 bytes) 2024-09-16T14:08:19.516 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:19.513+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:19.517 INFO:teuthology.orchestra.run.smithi079.stdout:3171 writesame 0xa39dece thru 0xa3aa49c data_size 0x1(0xc5cf bytes) 2024-09-16T14:08:19.538 INFO:teuthology.orchestra.run.smithi079.stdout:3172 write 0xedfff44 thru 0xee0534b (0x5408 bytes) 2024-09-16T14:08:19.558 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:19.558 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:19.566 INFO:teuthology.orchestra.run.smithi079.stdout:3173 trunc from 0xee0534c to 0x962bd7f 2024-09-16T14:08:19.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:19.588 INFO:teuthology.orchestra.run.smithi079.stdout:3174 punch from 0xf3e74f to 0xf4386f, (0x5120 bytes) 2024-09-16T14:08:19.588 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:19.588 INFO:teuthology.orchestra.run.smithi079.stdout:3176 trunc from 0x962bd7f to 0x955dd90 2024-09-16T14:08:19.601 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:19.601 INFO:teuthology.orchestra.run.smithi079.stdout:3178 punch from 0x28b9bb to 0x298131, (0xc776 bytes) 2024-09-16T14:08:19.601 INFO:teuthology.orchestra.run.smithi079.stdout:3179 read 0xfa7ba0 thru 0xfab5da (0x3a3b bytes) 2024-09-16T14:08:19.602 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:19.602 INFO:teuthology.orchestra.run.smithi079.stdout:3182 writesame 0x13d7785 thru 0x13e6228 data_size 0x2(0xeaa4 bytes) 2024-09-16T14:08:19.616 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:19.610+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:19.643 INFO:teuthology.orchestra.run.smithi079.stdout:3183 trunc from 0x955dd90 to 0x2f62380 2024-09-16T14:08:19.667 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:19.679 INFO:teuthology.orchestra.run.smithi079.stdout:3185 write 0x443149f thru 0x443a2d0 (0x8e32 bytes) 2024-09-16T14:08:19.689 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:19.690 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:19.697 INFO:teuthology.orchestra.run.smithi079.stdout:3186 trunc from 0x443a2d1 to 0x2dac174 2024-09-16T14:08:19.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:19.717 INFO:teuthology.orchestra.run.smithi079.stdout:3187 punch from 0x414e3d to 0x4247a7, (0xf96a bytes) 2024-09-16T14:08:19.717 INFO:teuthology.orchestra.run.smithi079.stdout:3188 read 0x6dde2a thru 0x6ea34d (0xc524 bytes) 2024-09-16T14:08:19.718 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:19.710+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:19.725 INFO:teuthology.orchestra.run.smithi079.stdout:3189 write 0x2bc5e30 thru 0x2bd55a3 (0xf774 bytes) 2024-09-16T14:08:19.805 INFO:teuthology.orchestra.run.smithi079.stdout:3193 writesame 0x3146848 thru 0x314a3d3 data_size 0x4a(0x3b8c bytes) 2024-09-16T14:08:19.805 INFO:teuthology.orchestra.run.smithi079.stdout:3194 punch from 0x18c1a77 to 0x18ce20b, (0xc794 bytes) 2024-09-16T14:08:19.805 INFO:teuthology.orchestra.run.smithi079.stdout:3195 read 0x1cd8de2 thru 0x1ce18be (0x8add bytes) 2024-09-16T14:08:19.819 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:19.814+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:19.822 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:19.822 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:19.830 INFO:teuthology.orchestra.run.smithi079.stdout:3196 writesame 0xab3b435 thru 0xab4a7f4 data_size 0x30(0xf3c0 bytes) 2024-09-16T14:08:19.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:19.847 INFO:teuthology.orchestra.run.smithi079.stdout:3197 write 0xdeba190 thru 0xdec7a3c (0xd8ad bytes) 2024-09-16T14:08:19.875 INFO:teuthology.orchestra.run.smithi079.stdout:3198 trunc from 0xdec7a3d to 0x8c09a3d 2024-09-16T14:08:19.896 INFO:teuthology.orchestra.run.smithi079.stdout:3199 write 0x74d442 thru 0x75b7e8 (0xe3a7 bytes) 2024-09-16T14:08:19.908 INFO:teuthology.orchestra.run.smithi079.stdout:3200 trunc from 0x8c09a3d to 0x53e178f 2024-09-16T14:08:19.920 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:19.917+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:19.920 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:19.920 INFO:teuthology.orchestra.run.smithi079.stdout:3202 read 0x2afd996 thru 0x2b0ab64 (0xd1cf bytes) 2024-09-16T14:08:19.946 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:19.946 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:19.948 INFO:teuthology.orchestra.run.smithi079.stdout:3203 write 0xc582e47 thru 0xc58bba7 (0x8d61 bytes) 2024-09-16T14:08:19.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:19.978 INFO:teuthology.orchestra.run.smithi079.stdout:3204 write 0xdeaf714 thru 0xdeb4f57 (0x5844 bytes) 2024-09-16T14:08:19.995 INFO:teuthology.orchestra.run.smithi079.stdout:3205 write 0x34d650a thru 0x34e23dd (0xbed4 bytes) 2024-09-16T14:08:20.021 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:20.014+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:20.031 INFO:teuthology.orchestra.run.smithi079.stdout:3206 trunc from 0xdeb4f58 to 0x229f848 2024-09-16T14:08:20.060 INFO:teuthology.orchestra.run.smithi079.stdout:3207 trunc from 0x229f848 to 0x4079045 2024-09-16T14:08:20.085 INFO:teuthology.orchestra.run.smithi079.stdout:3208 write 0x647192c thru 0x647a404 (0x8ad9 bytes) 2024-09-16T14:08:20.086 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:20.086 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:20.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:20.101 INFO:teuthology.orchestra.run.smithi079.stdout:3209 read 0x24fa9e2 thru 0x250457f (0x9b9e bytes) 2024-09-16T14:08:20.117 INFO:teuthology.orchestra.run.smithi079.stdout:3211 writesame 0x99b8a38 thru 0x99c5c9d data_size 0x5e(0xd266 bytes) 2024-09-16T14:08:20.118 INFO:teuthology.orchestra.run.smithi079.stdout:3212 read 0x129b308 thru 0x129c474 (0x116d bytes) 2024-09-16T14:08:20.122 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:20.114+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:20.134 INFO:teuthology.orchestra.run.smithi079.stdout:3213 trunc from 0x99c5c9e to 0x49e9ac 2024-09-16T14:08:20.223 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:20.218+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:20.225 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:20.225 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:20.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:20.243 INFO:teuthology.orchestra.run.smithi079.stdout:3214 read 0x1a8e13 thru 0x1a999c (0xb8a bytes) 2024-09-16T14:08:20.245 INFO:teuthology.orchestra.run.smithi079.stdout:3215 punch from 0x3e2bc2 to 0x3f1ac2, (0xef00 bytes) 2024-09-16T14:08:20.264 INFO:teuthology.orchestra.run.smithi079.stdout:3216 write 0x4820d19 thru 0x48225df (0x18c7 bytes) 2024-09-16T14:08:20.273 INFO:teuthology.orchestra.run.smithi079.stdout:3217 read 0x17f4aa8 thru 0x17fe545 (0x9a9e bytes) 2024-09-16T14:08:20.273 INFO:teuthology.orchestra.run.smithi079.stdout:3218 punch from 0x19f5422 to 0x19fa369, (0x4f47 bytes) 2024-09-16T14:08:20.273 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:20.281 INFO:teuthology.orchestra.run.smithi079.stdout:3221 trunc from 0x48225e0 to 0x95ae124 2024-09-16T14:08:20.294 INFO:teuthology.orchestra.run.smithi079.stdout:3222 read 0x367ae6f thru 0x367c8a1 (0x1a33 bytes) 2024-09-16T14:08:20.294 INFO:teuthology.orchestra.run.smithi079.stdout:3223 clone 30 order 22 su 65536 sc 2 2024-09-16T14:08:20.324 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:20.318+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:20.339 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:20.339 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:20.349 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:20.424 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:20.418+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:20.457 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:20.457 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:20.467 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:20.525 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:20.518+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:20.564 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:20.564 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:20.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:20.627 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:20.625+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:20.715 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:20.715 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:20.725 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:20.727 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:20.725+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:20.827 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:20.827 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:20.828 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:20.822+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:20.836 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:20.928 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:20.925+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:20.934 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:20.934 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:20.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:21.030 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:21.029+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:21.044 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:21.044 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:21.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:21.124 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-16T14:08:21.124 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:08:21.125 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-16T14:08:21.125 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T14:08:21.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T14:08:21.127 INFO:tasks.ceph.osd.3:Started 2024-09-16T14:08:21.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:21.130 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:21.126+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:21.160 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:21.160 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:21.170 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:21.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:08:21.230 INFO:teuthology.orchestra.run.smithi079.stderr:no valid command found; 10 closest matches: 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:0 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:1 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:2 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:abort 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:assert 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:config diff 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:config diff get 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:config get 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:config help [] 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:config set ... 2024-09-16T14:08:21.231 INFO:teuthology.orchestra.run.smithi079.stderr:admin_socket: invalid command 2024-09-16T14:08:21.232 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:21.226+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:21.232 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T14:08:21.233 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-16T14:08:21.257 INFO:teuthology.orchestra.run.smithi079.stdout:leaving image image_client.0-clone29 intact 2024-09-16T14:08:21.264 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:21.264 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:21.274 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:21.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:21.332 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:21.329+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:21.365 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:21.366 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:21.374 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:21.432 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:21.429+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:21.434 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-16T14:08:21.465 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:21.472 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:21.473 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:21.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:21.484 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:21.504 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:21.526 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:21.532 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:21.526+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:21.551 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:21.572 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:21.586 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:21.586 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:21.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:21.595 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:21.597 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:21.598 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:21.646 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:21.645+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.279217156000001, 2024-09-16T14:08:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:08:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:08:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:08:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.441254833000002, 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:08:21.699 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:08:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:08:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9140158449999998, 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:08:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9139926140000001, 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:08:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:08:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:08:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9139696669999999, 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:08:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.913953158, 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:08:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.913935532, 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:08:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9080197879999998, 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:08:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:08:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9080023829999999, 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:08:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:08:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:08:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9076480170000001, 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:08:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9045041579999999, 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:08:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9044888719999999, 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:08:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9044724629999998, 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:08:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:08:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:08:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9043972660000001, 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:08:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:08:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:08:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:08:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:08:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9042997189999999, 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.904274478, 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:08:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:08:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.904245424, 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:08:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:08:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:08:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9009814350000003, 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9009632339999998, 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:08:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.900924056, 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:08:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:21.794 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:21.795 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:21.765+0000 7f3f1a52d540 -1 Falling back to public interface 2024-09-16T14:08:21.796 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:21.790+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:21.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:21.856 INFO:teuthology.orchestra.run.smithi079.stdout:3225 writesame 0xb356801 thru 0xb36415e data_size 0x1(0xd95e bytes) 2024-09-16T14:08:21.856 INFO:teuthology.orchestra.run.smithi079.stdout:3226 read 0xa1fec8f thru 0xa205e71 (0x71e3 bytes) 2024-09-16T14:08:21.871 INFO:teuthology.orchestra.run.smithi079.stdout:3227 trunc from 0xb36415f to 0x3d55355 2024-09-16T14:08:21.896 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:21.893+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:21.897 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:21.897 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:21.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:21.912 INFO:teuthology.orchestra.run.smithi079.stdout:3228 write 0x6def5dc thru 0x6dfa1f8 (0xac1d bytes) 2024-09-16T14:08:21.921 INFO:teuthology.orchestra.run.smithi079.stdout:3231 read 0x3f879b0 thru 0x3f9225c (0xa8ad bytes) 2024-09-16T14:08:21.922 INFO:teuthology.orchestra.run.smithi079.stdout:3232 writesame 0x5a7fb93 thru 0x5a8061b data_size 0x1f(0xa89 bytes) 2024-09-16T14:08:21.932 INFO:teuthology.orchestra.run.smithi079.stdout:3235 writesame 0x5ca1fc3 thru 0x5ca4cc9 data_size 0x1(0x2d07 bytes) 2024-09-16T14:08:21.938 INFO:teuthology.orchestra.run.smithi079.stdout:3236 punch from 0x3a0af53 to 0x3a173cf, (0xc47c bytes) 2024-09-16T14:08:21.938 INFO:teuthology.orchestra.run.smithi079.stdout:3237 read 0x353e91e thru 0x35422a4 (0x3987 bytes) 2024-09-16T14:08:21.940 INFO:teuthology.orchestra.run.smithi079.stdout:3239 read 0x42e9c60 thru 0x42ec1bc (0x255d bytes) 2024-09-16T14:08:21.945 INFO:teuthology.orchestra.run.smithi079.stdout:3241 trunc from 0x6dfa1f9 to 0x8bbe173 2024-09-16T14:08:21.984 INFO:teuthology.orchestra.run.smithi079.stdout:3245 writesame 0xc69922d thru 0xc69df06 data_size 0x9(0x4cda bytes) 2024-09-16T14:08:21.984 INFO:teuthology.orchestra.run.smithi079.stdout:3247 write 0xbe2116b thru 0xbe2f14b (0xdfe1 bytes) 2024-09-16T14:08:21.993 INFO:teuthology.orchestra.run.smithi079.stdout:3248 read 0x7790429 thru 0x7795813 (0x53eb bytes) 2024-09-16T14:08:21.993 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:21.993 INFO:teuthology.orchestra.run.smithi079.stdout:3250 write 0x258669 thru 0x25c873 (0x420b bytes) 2024-09-16T14:08:21.996 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:21.990+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:22.025 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:22.026 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:22.026 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:22.026 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:22.026 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:22.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:22.040 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.040 INFO:teuthology.orchestra.run.smithi079.stdout:3254 read 0x8fab25b thru 0x8fad2f6 (0x209c bytes) 2024-09-16T14:08:22.040 INFO:teuthology.orchestra.run.smithi079.stdout:3255 read 0xb1272c2 thru 0xb132a07 (0xb746 bytes) 2024-09-16T14:08:22.040 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.040 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.040 INFO:teuthology.orchestra.run.smithi079.stdout:3258 read 0x8bb0e0 thru 0x8bce8c (0x1dad bytes) 2024-09-16T14:08:22.043 INFO:teuthology.orchestra.run.smithi079.stdout:3259 writesame 0x3c0da28 thru 0x3c1bc9a data_size 0x1(0xe273 bytes) 2024-09-16T14:08:22.073 INFO:teuthology.orchestra.run.smithi079.stdout:3261 read 0x8c0738f thru 0x8c0c7a3 (0x5415 bytes) 2024-09-16T14:08:22.073 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.073 INFO:teuthology.orchestra.run.smithi079.stdout:3263 punch from 0x65c96a1 to 0x65d88a6, (0xf205 bytes) 2024-09-16T14:08:22.073 INFO:teuthology.orchestra.run.smithi079.stdout:3264 trunc from 0xc69df07 to 0xc0c3028 2024-09-16T14:08:22.086 INFO:teuthology.orchestra.run.smithi079.stdout:3265 punch from 0xbb1ad20 to 0xbb27e2c, (0xd10c bytes) 2024-09-16T14:08:22.086 INFO:teuthology.orchestra.run.smithi079.stdout:3266 writesame 0xc0341ae thru 0xc03ec95 data_size 0xc(0xaae8 bytes) 2024-09-16T14:08:22.097 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:22.093+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:22.111 INFO:teuthology.orchestra.run.smithi079.stdout:3267 write 0xec155d2 thru 0xec20906 (0xb335 bytes) 2024-09-16T14:08:22.121 INFO:teuthology.orchestra.run.smithi079.stdout:3268 write 0x921b207 thru 0x921df9d (0x2d97 bytes) 2024-09-16T14:08:22.126 INFO:teuthology.orchestra.run.smithi079.stdout:3269 write 0x5cbfdcb thru 0x5cc1b12 (0x1d48 bytes) 2024-09-16T14:08:22.129 INFO:teuthology.orchestra.run.smithi079.stdout:3270 write 0x24f3a83 thru 0x24ff639 (0xbbb7 bytes) 2024-09-16T14:08:22.139 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:22.139 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226436079999999, 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:22.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:22.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:22.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:22.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226426353000001, 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:22.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226402790000002, 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.151 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:22.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:22.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:22.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226375551, 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:22.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:22.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:22.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:22.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226366137999999, 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.164 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:22.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:22.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:22.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.169 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226341819000002, 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.170 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.175 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226322213, 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:22.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:22.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:22.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:22.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:22.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:22.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:22.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:22.182 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226302024999999, 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.183 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:22.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226278755999999, 2024-09-16T14:08:22.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:22.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:22.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226248506000001, 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:22.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:22.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226231754000001, 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:22.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:22.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.203 INFO:teuthology.orchestra.run.smithi079.stdout:3273 read 0x1537bf1 thru 0x15426b2 (0xaac2 bytes) 2024-09-16T14:08:22.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:22.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:22.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:22.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:22.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.22621449, 2024-09-16T14:08:22.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:22.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226175653999999, 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:22.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:22.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226140826000002, 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:22.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226104939999999, 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:22.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:22.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:22.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226082347999998, 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.227 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:22.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226053275999998, 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.232 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:22.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.233 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:22.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:22.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:22.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:22.240 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226034930000001, 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:22.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:22.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226017591000002, 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:22.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:22.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.226001667999999, 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:22.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.256 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:22.257 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.257 INFO:teuthology.orchestra.run.smithi079.stdout:3275 read 0x3a52f80 thru 0x3a62ea3 (0xff24 bytes) 2024-09-16T14:08:22.257 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.257 INFO:teuthology.orchestra.run.smithi079.stdout:3277 writesame 0x84b7501 thru 0x84c605e data_size 0x2f(0xeb5e bytes) 2024-09-16T14:08:22.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:22.259 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:08:22.259 INFO:teuthology.orchestra.run.smithi079.stdout:3280 trunc from 0xec20907 to 0xa76de24 2024-09-16T14:08:22.261 INFO:teuthology.orchestra.run.smithi079.stdout:3282 write 0xe00e823 thru 0xe01e12b (0xf909 bytes) 2024-09-16T14:08:22.261 INFO:teuthology.orchestra.run.smithi079.stdout:3283 punch from 0x62e5f to 0x65c4a, (0x2deb bytes) 2024-09-16T14:08:22.261 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.261 INFO:teuthology.orchestra.run.smithi079.stdout:3286 write 0x8731fb0 thru 0x8732d6b (0xdbc bytes) 2024-09-16T14:08:22.261 INFO:teuthology.orchestra.run.smithi079.stdout:3287 punch from 0xbb1b5be to 0xbb1e315, (0x2d57 bytes) 2024-09-16T14:08:22.261 INFO:teuthology.orchestra.run.smithi079.stdout:3289 trunc from 0xe01e12c to 0x33fa3ad 2024-09-16T14:08:22.261 INFO:teuthology.orchestra.run.smithi079.stdout:3290 read 0x197bb1c thru 0x1986376 (0xa85b bytes) 2024-09-16T14:08:22.262 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:22.254+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:22.270 INFO:teuthology.orchestra.run.smithi079.stdout:3291 trunc from 0x33fa3ad to 0x6fe8824 2024-09-16T14:08:22.287 INFO:teuthology.orchestra.run.smithi079.stdout:3292 trunc from 0x6fe8824 to 0xa6be133 2024-09-16T14:08:22.304 INFO:teuthology.orchestra.run.smithi079.stdout:3293 trunc from 0xa6be133 to 0x845ecdb 2024-09-16T14:08:22.316 INFO:teuthology.orchestra.run.smithi079.stdout:3294 read 0x762c8a4 thru 0x762f456 (0x2bb3 bytes) 2024-09-16T14:08:22.316 INFO:teuthology.orchestra.run.smithi079.stdout:3296 punch from 0x4eefd55 to 0x4ef0b3f, (0xdea bytes) 2024-09-16T14:08:22.316 INFO:teuthology.orchestra.run.smithi079.stdout:3298 write 0x63dafd8 thru 0x63dfe9a (0x4ec3 bytes) 2024-09-16T14:08:22.324 INFO:teuthology.orchestra.run.smithi079.stdout:3301 read 0x4d7f70e thru 0x4d8e737 (0xf02a bytes) 2024-09-16T14:08:22.339 INFO:teuthology.orchestra.run.smithi079.stdout:3302 write 0x99c53cd thru 0x99c8302 (0x2f36 bytes) 2024-09-16T14:08:22.359 INFO:teuthology.orchestra.run.smithi079.stdout:3303 trunc from 0x99c8303 to 0x3b8cc4d 2024-09-16T14:08:22.361 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:22.354+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:22.364 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:22.364 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:22.365 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:22.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:22.381 INFO:teuthology.orchestra.run.smithi079.stdout:3304 punch from 0x24821c7 to 0x2488677, (0x64b0 bytes) 2024-09-16T14:08:22.399 INFO:teuthology.orchestra.run.smithi079.stdout:3305 trunc from 0x3b8cc4d to 0xe09dfd5 2024-09-16T14:08:22.410 INFO:teuthology.orchestra.run.smithi079.stdout:3307 read 0xda9c63c thru 0xdaa0dad (0x4772 bytes) 2024-09-16T14:08:22.410 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.410 INFO:teuthology.orchestra.run.smithi079.stdout:3309 read 0xe01c471 thru 0xe01cd23 (0x8b3 bytes) 2024-09-16T14:08:22.410 INFO:teuthology.orchestra.run.smithi079.stdout:3311 punch from 0xd7df630 to 0xd7e60c2, (0x6a92 bytes) 2024-09-16T14:08:22.410 INFO:teuthology.orchestra.run.smithi079.stdout:3312 write 0x1c486d6 thru 0x1c4c590 (0x3ebb bytes) 2024-09-16T14:08:22.414 INFO:teuthology.orchestra.run.smithi079.stdout:3313 punch from 0xb912687 to 0xb92254a, (0xfec3 bytes) 2024-09-16T14:08:22.414 INFO:teuthology.orchestra.run.smithi079.stdout:3314 punch from 0x63e1a41 to 0x63e3774, (0x1d33 bytes) 2024-09-16T14:08:22.414 INFO:teuthology.orchestra.run.smithi079.stdout:3315 write 0x4b23c7 thru 0x4c210f (0xfd49 bytes) 2024-09-16T14:08:22.463 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:22.458+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:22.470 INFO:teuthology.orchestra.run.smithi079.stdout:3316 write 0x16dacc3 thru 0x16df4c0 (0x47fe bytes) 2024-09-16T14:08:22.473 INFO:teuthology.orchestra.run.smithi079.stdout:3317 write 0xd3a8ea1 thru 0xd3adfe7 (0x5147 bytes) 2024-09-16T14:08:22.481 INFO:teuthology.orchestra.run.smithi079.stdout:3318 writesame 0x6bb4af4 thru 0x6bc02d3 data_size 0x10(0xb7e0 bytes) 2024-09-16T14:08:22.492 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:22.492 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:22.492 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:22.493 INFO:teuthology.orchestra.run.smithi079.stdout:3319 trunc from 0xe09dfd5 to 0xe6c658e 2024-09-16T14:08:22.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:22.506 INFO:teuthology.orchestra.run.smithi079.stdout:3320 writesame 0xaa2d99c thru 0xaa2f068 data_size 0x1(0x16cd bytes) 2024-09-16T14:08:22.515 INFO:teuthology.orchestra.run.smithi079.stdout:3321 write 0xbfed621 thru 0xbff8901 (0xb2e1 bytes) 2024-09-16T14:08:22.525 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.526 INFO:teuthology.orchestra.run.smithi079.stdout:3323 read 0x78ea2a6 thru 0x78f26a7 (0x8402 bytes) 2024-09-16T14:08:22.526 INFO:teuthology.orchestra.run.smithi079.stdout:3324 write 0xb30084 thru 0xb3a418 (0xa395 bytes) 2024-09-16T14:08:22.529 INFO:teuthology.orchestra.run.smithi079.stdout:3326 punch from 0x1675271 to 0x167cb55, (0x78e4 bytes) 2024-09-16T14:08:22.529 INFO:teuthology.orchestra.run.smithi079.stdout:3327 write 0x157c4d0 thru 0x158be26 (0xf957 bytes) 2024-09-16T14:08:22.533 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.533 INFO:teuthology.orchestra.run.smithi079.stdout:3329 read 0xabc82ba thru 0xabcf919 (0x7660 bytes) 2024-09-16T14:08:22.533 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.533 INFO:teuthology.orchestra.run.smithi079.stdout:3334 read 0x1777591 thru 0x1777b21 (0x591 bytes) 2024-09-16T14:08:22.557 INFO:teuthology.orchestra.run.smithi079.stdout:3336 trunc from 0xe6c658e to 0x46bd7b 2024-09-16T14:08:22.564 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:22.561+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:22.601 INFO:teuthology.orchestra.run.smithi079.stdout:3337 write 0x4cf794b thru 0x4d00ad5 (0x918b bytes) 2024-09-16T14:08:22.611 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:22.611 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:22.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.705317569999998, 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.612 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:08:22.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:08:22.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.617 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.705294861999999, 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:08:22.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:08:22.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.705270710000001, 2024-09-16T14:08:22.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:08:22.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.705237769, 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.629 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:22.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.705213086000001, 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.634 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:08:22.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.639 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.705189857000001, 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:22.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:22.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:08:22.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:08:22.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:08:22.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:08:22.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:08:22.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.647 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.703343893, 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:08:22.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:08:22.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.703301028999999, 2024-09-16T14:08:22.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:08:22.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:08:22.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701841411, 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.664 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701825641999999, 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:22.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:08:22.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.70180989, 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.670 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:22.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701794458999998, 2024-09-16T14:08:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:08:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701778821000001, 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:08:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701763681999999, 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:08:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:08:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701748845000001, 2024-09-16T14:08:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:08:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701725840999998, 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:08:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:08:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701710616, 2024-09-16T14:08:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:08:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701695432000001, 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:08:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:08:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701680187000001, 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:08:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.701665029000001, 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:08:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:22.725 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:22.727 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:22.728 INFO:teuthology.orchestra.run.smithi079.stdout:3338 write 0xc90f789 thru 0xc91a1fc (0xaa74 bytes) 2024-09-16T14:08:22.728 INFO:teuthology.orchestra.run.smithi079.stdout:3339 punch from 0x729d556 to 0x729e257, (0xd01 bytes) 2024-09-16T14:08:22.728 INFO:teuthology.orchestra.run.smithi079.stdout:3341 write 0xc02cb2e thru 0xc03b554 (0xea27 bytes) 2024-09-16T14:08:22.728 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.728 INFO:teuthology.orchestra.run.smithi079.stdout:3343 write 0x84d0232 thru 0x84d35b6 (0x3385 bytes) 2024-09-16T14:08:22.728 INFO:teuthology.orchestra.run.smithi079.stdout:3347 read 0x6f5052f thru 0x6f5519b (0x4c6d bytes) 2024-09-16T14:08:22.728 INFO:teuthology.orchestra.run.smithi079.stdout:3348 write 0x934374 thru 0x937b1e (0x37ab bytes) 2024-09-16T14:08:22.729 INFO:teuthology.orchestra.run.smithi079.stdout:3349 read 0x6b078e4 thru 0x6b0bc7f (0x439c bytes) 2024-09-16T14:08:22.729 INFO:teuthology.orchestra.run.smithi079.stdout:3351 trunc from 0xc91a1fd to 0x465a091 2024-09-16T14:08:22.729 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:22.725+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:22.740 INFO:teuthology.orchestra.run.smithi079.stdout:3352 writesame 0xc55231c thru 0xc55f40a data_size 0x1b(0xd0ef bytes) 2024-09-16T14:08:22.740 INFO:teuthology.orchestra.run.smithi079.stdout:3353 trunc from 0xc55f40b to 0xc6c12f2 2024-09-16T14:08:22.752 INFO:teuthology.orchestra.run.smithi079.stdout:3354 write 0x279cfa6 thru 0x27a33f7 (0x6452 bytes) 2024-09-16T14:08:22.761 INFO:teuthology.orchestra.run.smithi079.stdout:3355 read 0x8702171 thru 0x8709ecb (0x7d5b bytes) 2024-09-16T14:08:22.768 INFO:teuthology.orchestra.run.smithi079.stdout:3356 trunc from 0xc6c12f2 to 0x96091db 2024-09-16T14:08:22.782 INFO:teuthology.orchestra.run.smithi079.stdout:3358 read 0x6c75cf thru 0x6caeb7 (0x38e9 bytes) 2024-09-16T14:08:22.783 INFO:teuthology.orchestra.run.smithi079.stdout:3361 writesame 0x13d9ab7 thru 0x13e7dd5 data_size 0x1(0xe31f bytes) 2024-09-16T14:08:22.808 INFO:teuthology.orchestra.run.smithi079.stdout:3362 writesame 0x525f22e thru 0x5264b0b data_size 0x23(0x58de bytes) 2024-09-16T14:08:22.821 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.821 INFO:teuthology.orchestra.run.smithi079.stdout:3364 read 0x90eb32f thru 0x90f510f (0x9de1 bytes) 2024-09-16T14:08:22.821 INFO:teuthology.orchestra.run.smithi079.stdout:3366 trunc from 0x96091db to 0x9bc1206 2024-09-16T14:08:22.827 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:22.825+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:22.832 INFO:teuthology.orchestra.run.smithi079.stdout:3367 read 0x73b3f7d thru 0x73c22cf (0xe353 bytes) 2024-09-16T14:08:22.833 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.833 INFO:teuthology.orchestra.run.smithi079.stdout:3369 punch from 0x2a9afb7 to 0x2a9d660, (0x26a9 bytes) 2024-09-16T14:08:22.839 INFO:teuthology.orchestra.run.smithi079.stdout:3371 trunc from 0x9bc1206 to 0x55eac82 2024-09-16T14:08:22.851 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:22.852 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:22.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:22.882 INFO:teuthology.orchestra.run.smithi079.stdout:3373 writesame 0xd992529 thru 0xd998031 data_size 0x3b(0x5b09 bytes) 2024-09-16T14:08:22.882 INFO:teuthology.orchestra.run.smithi079.stdout:3374 punch from 0x1ac47f5 to 0x1ad02f0, (0xbafb bytes) 2024-09-16T14:08:22.882 INFO:teuthology.orchestra.run.smithi079.stdout:3375 punch from 0x897a6cd to 0x898177a, (0x70ad bytes) 2024-09-16T14:08:22.882 INFO:teuthology.orchestra.run.smithi079.stdout:3376 write 0x70eb3ce thru 0x70eca36 (0x1669 bytes) 2024-09-16T14:08:22.890 INFO:teuthology.orchestra.run.smithi079.stdout:3377 write 0xfa6996 thru 0xfad5a7 (0x6c12 bytes) 2024-09-16T14:08:22.896 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.897 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:22.926 INFO:teuthology.orchestra.run.smithi079.stdout:3380 trunc from 0xd998032 to 0xb504a6 2024-09-16T14:08:22.928 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:22.922+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:22.953 INFO:teuthology.orchestra.run.smithi079.stdout:3383 trunc from 0xb504a6 to 0x1e82121 2024-09-16T14:08:22.972 INFO:teuthology.orchestra.run.smithi079.stdout:3384 trunc from 0x1e82121 to 0x64d3137 2024-09-16T14:08:22.984 INFO:teuthology.orchestra.run.smithi079.stdout:3385 punch from 0x54a9aa6 to 0x54b62d7, (0xc831 bytes) 2024-09-16T14:08:22.991 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:22.991 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:23.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:23.014 INFO:teuthology.orchestra.run.smithi079.stdout:3387 write 0xdf0eb12 thru 0xdf1485a (0x5d49 bytes) 2024-09-16T14:08:23.023 INFO:teuthology.orchestra.run.smithi079.stdout:3388 write 0xaba8c62 thru 0xabb24f7 (0x9896 bytes) 2024-09-16T14:08:23.029 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:23.025+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:23.032 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.032 INFO:teuthology.orchestra.run.smithi079.stdout:3391 write 0xb2a8370 thru 0xb2b7947 (0xf5d8 bytes) 2024-09-16T14:08:23.041 INFO:teuthology.orchestra.run.smithi079.stdout:3393 writesame 0x75e3149 thru 0x75e4b2a data_size 0x1(0x19e2 bytes) 2024-09-16T14:08:23.052 INFO:teuthology.orchestra.run.smithi079.stdout:3395 write 0xb10a87b thru 0xb11a51d (0xfca3 bytes) 2024-09-16T14:08:23.054 INFO:teuthology.orchestra.run.smithi079.stdout:3396 write 0x2a3966a thru 0x2a45f4a (0xc8e1 bytes) 2024-09-16T14:08:23.063 INFO:teuthology.orchestra.run.smithi079.stdout:3397 read 0x4a37cc5 thru 0x4a3c8b1 (0x4bed bytes) 2024-09-16T14:08:23.076 INFO:teuthology.orchestra.run.smithi079.stdout:3398 write 0xeb7c611 thru 0xeb8a51b (0xdf0b bytes) 2024-09-16T14:08:23.086 INFO:teuthology.orchestra.run.smithi079.stdout:3400 write 0xf58c85 thru 0xf67fc3 (0xf33f bytes) 2024-09-16T14:08:23.090 INFO:teuthology.orchestra.run.smithi079.stdout:3401 write 0x530cd20 thru 0x5319fa3 (0xd284 bytes) 2024-09-16T14:08:23.099 INFO:teuthology.orchestra.run.smithi079.stdout:3402 write 0x7ba99cb thru 0x7bb8ccf (0xf305 bytes) 2024-09-16T14:08:23.107 INFO:teuthology.orchestra.run.smithi079.stdout:3403 read 0xc79f03 thru 0xc7ed45 (0x4e43 bytes) 2024-09-16T14:08:23.108 INFO:teuthology.orchestra.run.smithi079.stdout:3404 write 0x307aade thru 0x3089154 (0xe677 bytes) 2024-09-16T14:08:23.112 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:23.112 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86627 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 115712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.938132406000001, 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057990535000000003, 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86627 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.79e-07 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172073+0000", 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.53e-06 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172077+0000", 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172079+0000", 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224325+0000", 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052246133 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224394+0000", 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8960000000000004e-05 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.230062+0000", 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056682820000000002 2024-09-16T14:08:23.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86989 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.916523560000002, 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058737710999999998, 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.118 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86989 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193681+0000", 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193682+0000", 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6169999999999999e-06 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193687+0000", 2024-09-16T14:08:23.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7210000000000002e-06 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193690+0000", 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.072e-06 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.246881+0000", 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.053191117000000003 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252309+0000", 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054275979999999996 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252418+0000", 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109255 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.122 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86995 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.916497358000001, 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059640662999999997, 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86995 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:23.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:23.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197498+0000", 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037915219999999999 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197504+0000", 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7520000000000002e-06 2024-09-16T14:08:23.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197515+0000", 2024-09-16T14:08:23.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.117e-05 2024-09-16T14:08:23.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197521+0000", 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.083e-06 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253121+0000", 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055600254000000002 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253227+0000", 2024-09-16T14:08:23.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010607000000000001 2024-09-16T14:08:23.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253347+0000", 2024-09-16T14:08:23.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119812 2024-09-16T14:08:23.129 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.129 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87017 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912561621999998, 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676182999999998, 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.130 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87017 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197643+0000", 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197645+0000", 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197652+0000", 2024-09-16T14:08:23.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8430000000000004e-06 2024-09-16T14:08:23.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197656+0000", 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254046+0000", 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056390033999999999 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254205+0000", 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158639 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254318+0000", 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113916 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87019 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912544046000001, 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056580459, 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87019 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.136 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197663+0000", 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.396e-06 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197669+0000", 2024-09-16T14:08:23.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4099999999999996e-06 2024-09-16T14:08:23.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197673+0000", 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254066+0000", 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056392870999999997 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254160+0000", 2024-09-16T14:08:23.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4574000000000006e-05 2024-09-16T14:08:23.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254240+0000", 2024-09-16T14:08:23.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0125999999999998e-05 2024-09-16T14:08:23.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.142 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.142 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87023 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 193536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.142 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:23.143 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912527179000001, 2024-09-16T14:08:23.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056736048999999997, 2024-09-16T14:08:23.143 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.143 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.144 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87023 2024-09-16T14:08:23.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.144 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:23.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:23.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:23.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:08:23.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197679+0000", 2024-09-16T14:08:23.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-16T14:08:23.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197686+0000", 2024-09-16T14:08:23.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-16T14:08:23.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197690+0000", 2024-09-16T14:08:23.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.177e-06 2024-09-16T14:08:23.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254263+0000", 2024-09-16T14:08:23.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056573364000000001 2024-09-16T14:08:23.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254334+0000", 2024-09-16T14:08:23.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0574000000000006e-05 2024-09-16T14:08:23.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254413+0000", 2024-09-16T14:08:23.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079999999999995e-05 2024-09-16T14:08:23.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.151 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.152 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87027 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.152 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:23.153 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912510055999999, 2024-09-16T14:08:23.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056974745, 2024-09-16T14:08:23.153 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.153 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.154 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87027 2024-09-16T14:08:23.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.154 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:23.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:23.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:23.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2599999999999998e-07 2024-09-16T14:08:23.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197696+0000", 2024-09-16T14:08:23.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1679999999999998e-06 2024-09-16T14:08:23.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197703+0000", 2024-09-16T14:08:23.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4389999999999997e-06 2024-09-16T14:08:23.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197707+0000", 2024-09-16T14:08:23.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:08:23.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254383+0000", 2024-09-16T14:08:23.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676509999999999 2024-09-16T14:08:23.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254483+0000", 2024-09-16T14:08:23.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9692999999999996e-05 2024-09-16T14:08:23.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254669+0000", 2024-09-16T14:08:23.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018580300000000001 2024-09-16T14:08:23.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.162 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.163 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87029 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 197632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.163 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:23.163 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912493219000002, 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057332283999999997, 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87029 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:23.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197713+0000", 2024-09-16T14:08:23.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.199e-06 2024-09-16T14:08:23.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197719+0000", 2024-09-16T14:08:23.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.699e-06 2024-09-16T14:08:23.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197723+0000", 2024-09-16T14:08:23.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000002e-06 2024-09-16T14:08:23.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254709+0000", 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056986536999999997 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254790+0000", 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0192999999999996e-05 2024-09-16T14:08:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255043+0000", 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025339500000000001 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87031 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 234496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:23.169 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912476927, 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057120535, 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87031 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:23.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0600000000000001e-07 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197729+0000", 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0499999999999999e-06 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197736+0000", 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2249999999999997e-06 2024-09-16T14:08:23.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197739+0000", 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6839999999999998e-06 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254697+0000", 2024-09-16T14:08:23.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056957902999999997 2024-09-16T14:08:23.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254776+0000", 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254848+0000", 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1044999999999995e-05 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.176 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87035 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 198656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.176 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:23.176 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912460923000001, 2024-09-16T14:08:23.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057340290000000002, 2024-09-16T14:08:23.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87035 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:23.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197744+0000", 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6499999999999999e-07 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197746+0000", 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0169999999999999e-06 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197752+0000", 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1040000000000003e-06 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197755+0000", 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.782e-06 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254869+0000", 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057113766000000003 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254971+0000", 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101932 2024-09-16T14:08:23.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255083+0000", 2024-09-16T14:08:23.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112124 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87039 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 238592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912444513000001, 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057586228000000003, 2024-09-16T14:08:23.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87039 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:23.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197760+0000", 2024-09-16T14:08:23.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197762+0000", 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.829e-06 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197768+0000", 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5470000000000002e-06 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197772+0000", 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4699999999999998e-06 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255109+0000", 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057337232000000002 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255184+0000", 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5593000000000006e-05 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255346+0000", 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016120899999999999 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87041 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 202752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912428459000001, 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057827176000000001, 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87041 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197776+0000", 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:23.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197778+0000", 2024-09-16T14:08:23.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3709999999999998e-06 2024-09-16T14:08:23.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197784+0000", 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.054e-06 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197787+0000", 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1879999999999998e-06 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255397+0000", 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057609059999999997 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255483+0000", 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6415999999999994e-05 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255603+0000", 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119703 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87043 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 203776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912412291999999, 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057723202000000001, 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87043 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197801+0000", 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5869999999999993e-06 2024-09-16T14:08:23.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197807+0000", 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197811+0000", 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0099999999999997e-06 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255377+0000", 2024-09-16T14:08:23.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057566063000000001 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255450+0000", 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3192e-05 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255515+0000", 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4543999999999997e-05 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87045 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912388099000001, 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057864405000000001, 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87045 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.199 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197819+0000", 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.311e-06 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197824+0000", 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6980000000000003e-06 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197828+0000", 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2339999999999999e-06 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255537+0000", 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057709979000000002 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255613+0000", 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5807000000000001e-05 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255680+0000", 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6935999999999997e-05 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87051 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.203 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.912372947000001, 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058010645999999999, 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87051 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:23.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200527+0000", 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026964990000000002 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200532+0000", 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8899999999999998e-06 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200541+0000", 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1249999999999993e-06 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200546+0000", 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4129999999999997e-06 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255630+0000", 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055083612999999997 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255711+0000", 2024-09-16T14:08:23.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1934000000000005e-05 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255842+0000", 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013017200000000001 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87559 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 100352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:23.208 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.909538591, 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056509688000000002, 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87559 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200666+0000", 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200668+0000", 2024-09-16T14:08:23.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.21e-06 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200674+0000", 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.896e-06 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200678+0000", 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7050000000000001e-06 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.256898+0000", 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056220883999999999 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257014+0000", 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011530499999999999 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257175+0000", 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016127200000000001 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87568 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.909506612000001, 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056739408999999998, 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87568 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.213 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200698+0000", 2024-09-16T14:08:23.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200700+0000", 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.272e-06 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200706+0000", 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7080000000000002e-06 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200709+0000", 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.394e-06 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257221+0000", 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056512383999999999 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257319+0000", 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7510000000000007e-05 2024-09-16T14:08:23.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257437+0000", 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011773399999999999 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87570 3.e 3:73b2712a:::rbd_data.113619b3ee73.0000000000000027:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.909491280000001, 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056717201000000002, 2024-09-16T14:08:23.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87570 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201471+0000", 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075828899999999999 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201480+0000", 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.448e-06 2024-09-16T14:08:23.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201494+0000", 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3162999999999999e-05 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201500+0000", 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0090000000000004e-06 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257205+0000", 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055705603999999999 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257330+0000", 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124972 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257430+0000", 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9716000000000005e-05 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87651 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 136192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.904591574000001, 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056431633000000002, 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87651 2024-09-16T14:08:23.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205612+0000", 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205613+0000", 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3999999999999998e-07 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205614+0000", 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205619+0000", 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.758e-06 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.205622+0000", 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0690000000000002e-06 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.261887+0000", 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056265241000000001 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.261965+0000", 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7671999999999998e-05 2024-09-16T14:08:23.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262044+0000", 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9320999999999995e-05 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87773 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 63488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.903714266000001, 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056458201, 2024-09-16T14:08:23.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87773 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207001+0000", 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051104000000000004 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207006+0000", 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207019+0000", 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2266000000000001e-05 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207025+0000", 2024-09-16T14:08:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2029999999999997e-06 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262711+0000", 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055686539 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262801+0000", 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9895999999999994e-05 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:23.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262948+0000", 2024-09-16T14:08:23.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146642 2024-09-16T14:08:23.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.231 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:23.231 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:23.231 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:23.232 INFO:teuthology.orchestra.run.smithi079.stdout:3405 read 0x8353fe4 thru 0x83581ee (0x420b bytes) 2024-09-16T14:08:23.232 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.233 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:23.234 INFO:teuthology.orchestra.run.smithi079.stdout:3407 trunc from 0xeb8a51c to 0x24aef8 2024-09-16T14:08:23.234 INFO:teuthology.orchestra.run.smithi079.stdout:3408 read 0x174271 thru 0x18219f (0xdf2f bytes) 2024-09-16T14:08:23.234 INFO:teuthology.orchestra.run.smithi079.stdout:3409 write 0x29596ff thru 0x295f9c9 (0x62cb bytes) 2024-09-16T14:08:23.234 INFO:teuthology.orchestra.run.smithi079.stdout:3411 read 0x8cca8 thru 0x96249 (0x95a2 bytes) 2024-09-16T14:08:23.234 INFO:teuthology.orchestra.run.smithi079.stdout:3413 trunc from 0x295f9ca to 0xd7b3e65 2024-09-16T14:08:23.234 INFO:teuthology.orchestra.run.smithi079.stdout:3416 write 0x10963c6 thru 0x109c918 (0x6553 bytes) 2024-09-16T14:08:23.234 INFO:teuthology.orchestra.run.smithi079.stdout:3418 writesame 0x96f0733 thru 0x96fc08c data_size 0x2(0xb95a bytes) 2024-09-16T14:08:23.235 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:23.229+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:23.259 INFO:teuthology.orchestra.run.smithi079.stdout:3419 read 0x64342db thru 0x6435642 (0x1368 bytes) 2024-09-16T14:08:23.259 INFO:teuthology.orchestra.run.smithi079.stdout:3420 read 0x771ea8c thru 0x7720e9b (0x2410 bytes) 2024-09-16T14:08:23.259 INFO:teuthology.orchestra.run.smithi079.stdout:3421 read 0x28ad739 thru 0x28b090e (0x31d6 bytes) 2024-09-16T14:08:23.259 INFO:teuthology.orchestra.run.smithi079.stdout:3422 write 0x8a401bf thru 0x8a444db (0x431d bytes) 2024-09-16T14:08:23.266 INFO:teuthology.orchestra.run.smithi079.stdout:3423 write 0x47b66a1 thru 0x47c50af (0xea0f bytes) 2024-09-16T14:08:23.275 INFO:teuthology.orchestra.run.smithi079.stdout:3424 read 0xcf46bb5 thru 0xcf5136c (0xa7b8 bytes) 2024-09-16T14:08:23.275 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.275 INFO:teuthology.orchestra.run.smithi079.stdout:3426 punch from 0x598b87a to 0x5991973, (0x60f9 bytes) 2024-09-16T14:08:23.275 INFO:teuthology.orchestra.run.smithi079.stdout:3427 write 0x1d51fc5 thru 0x1d5a7dd (0x8819 bytes) 2024-09-16T14:08:23.282 INFO:teuthology.orchestra.run.smithi079.stdout:3429 write 0x6244c96 thru 0x62474bd (0x2828 bytes) 2024-09-16T14:08:23.311 INFO:teuthology.orchestra.run.smithi079.stdout:3431 trunc from 0xd7b3e65 to 0x19ea352 2024-09-16T14:08:23.333 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:23.329+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:23.335 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:143872 3.9 3:920c7cb1:::rbd_header.11362441f8a0:head [call rbd.parent_get in=13b out=42b,call rbd.parent_overlap_get in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:23.348294+0000", 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00025994100000000002, 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029491599999999999, 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 143872 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:23.348294+0000", 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:23.348294+0000", 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:23.348295+0000", 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.609e-06 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:23.348303+0000", 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9980000000000003e-06 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:23.348327+0000", 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3802e-05 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:23.348336+0000", 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8529999999999998e-06 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:23.348387+0000", 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0421999999999999e-05 2024-09-16T14:08:23.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:23.348461+0000", 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4586999999999996e-05 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T14:08:23.354 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:23.355 INFO:teuthology.orchestra.run.smithi079.stdout:3435 write 0x4cb621f thru 0x4cb66e5 (0x4c7 bytes) 2024-09-16T14:08:23.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:23.364 INFO:teuthology.orchestra.run.smithi079.stdout:3436 trunc from 0x4cb66e6 to 0x76a9050 2024-09-16T14:08:23.373 INFO:teuthology.orchestra.run.smithi079.stdout:3437 write 0x3c83281 thru 0x3c8c69a (0x941a bytes) 2024-09-16T14:08:23.398 INFO:teuthology.orchestra.run.smithi079.stdout:3438 writesame 0xad0cd63 thru 0xad121ee data_size 0xe(0x548c bytes) 2024-09-16T14:08:23.418 INFO:teuthology.orchestra.run.smithi079.stdout:3439 write 0xe4b9260 thru 0xe4c5be6 (0xc987 bytes) 2024-09-16T14:08:23.426 INFO:teuthology.orchestra.run.smithi079.stdout:3440 write 0x1f577d4 thru 0x1f672b7 (0xfae4 bytes) 2024-09-16T14:08:23.432 INFO:teuthology.orchestra.run.smithi079.stdout:3441 write 0x8229c53 thru 0x8239216 (0xf5c4 bytes) 2024-09-16T14:08:23.433 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:23.429+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:23.440 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.440 INFO:teuthology.orchestra.run.smithi079.stdout:3444 punch from 0x517d502 to 0x518a095, (0xcb93 bytes) 2024-09-16T14:08:23.440 INFO:teuthology.orchestra.run.smithi079.stdout:3445 read 0x353c21d thru 0x353d5ea (0x13ce bytes) 2024-09-16T14:08:23.451 INFO:teuthology.orchestra.run.smithi079.stdout:3446 write 0xe5e1cf6 thru 0xe5eaa3a (0x8d45 bytes) 2024-09-16T14:08:23.454 INFO:teuthology.orchestra.run.smithi079.stdout:3448 write 0x1b706fa thru 0x1b727e1 (0x20e8 bytes) 2024-09-16T14:08:23.465 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:23.465 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:23.465 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:23.465 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:23.465 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:23.472 INFO:teuthology.orchestra.run.smithi079.stdout:3449 trunc from 0xe5eaa3b to 0x8376d7e 2024-09-16T14:08:23.474 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:23.497 INFO:teuthology.orchestra.run.smithi079.stdout:3450 read 0x7e1879b thru 0x7e276ba (0xef20 bytes) 2024-09-16T14:08:23.498 INFO:teuthology.orchestra.run.smithi079.stdout:3451 read 0x6ba1060 thru 0x6bac07c (0xb01d bytes) 2024-09-16T14:08:23.509 INFO:teuthology.orchestra.run.smithi079.stdout:3452 trunc from 0x8376d7e to 0x25f769f 2024-09-16T14:08:23.534 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:23.533+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:23.563 INFO:teuthology.orchestra.run.smithi079.stdout:3453 write 0xcd97394 thru 0xcda10df (0x9d4c bytes) 2024-09-16T14:08:23.573 INFO:teuthology.orchestra.run.smithi079.stdout:3454 punch from 0x43c7cd to 0x447794, (0xafc7 bytes) 2024-09-16T14:08:23.573 INFO:teuthology.orchestra.run.smithi079.stdout:3455 writesame 0xb446971 thru 0xb456516 data_size 0x1b(0xfba6 bytes) 2024-09-16T14:08:23.581 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:23.581 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:23.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:23.581 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.239885134000005, 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:23.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:23.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:23.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.239862736999996, 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:23.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:23.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:23.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.239256026000007, 2024-09-16T14:08:23.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:23.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:23.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:23.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:23.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.239181699, 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:23.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:23.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:23.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.268256907000001, 2024-09-16T14:08:23.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:23.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.640298326, 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:23.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:23.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:23.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.110137860000002, 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:23.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:23.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.462589329, 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:23.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:23.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.628 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101781 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916343535999999, 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019058275, 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.629 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101781 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660443+0000", 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660450+0000", 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999997e-06 2024-09-16T14:08:23.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660454+0000", 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0029999999999996e-06 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679188+0000", 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018734869000000001 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679271+0000", 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2721999999999998e-05 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679499+0000", 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227612 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101783 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.633 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916326357999999, 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019291800000000001, 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101783 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4499999999999997e-07 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660465+0000", 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4659999999999996e-06 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660471+0000", 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5250000000000002e-06 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660480+0000", 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5779999999999999e-06 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679530+0000", 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019050085000000001 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679633+0000", 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103341 2024-09-16T14:08:23.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679750+0000", 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011636 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916299088999999, 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:23.638 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:23.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916281898000001, 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.647 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916244881000001, 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:23.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916224204999999, 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:23.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:23.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:23.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916182025000001, 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:23.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.661 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916147800000001, 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:23.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916118084000001, 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:23.666 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:23.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:23.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.916098422000001, 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.671 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:23.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:23.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:23.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:23.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:23.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.907338296999999, 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:23.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8226117549999996, 2024-09-16T14:08:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:08:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:08:23.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:08:23.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.686 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:23.686 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:23.686 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:23.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:23.688 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:23.689 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:23.682+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:3456 read 0x74dad35 thru 0x74ea2eb (0xf5b7 bytes) 2024-09-16T14:08:23.689 INFO:teuthology.orchestra.run.smithi079.stdout:3457 trunc from 0xcda10e0 to 0x886e720 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:3458 trunc from 0x886e720 to 0x61ff213 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:3459 write 0xc477bd3 thru 0xc47e8c0 (0x6cee bytes) 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:3460 write 0x86ead23 thru 0x86eff2f (0x520d bytes) 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:3461 write 0x1a1f5f2 thru 0x1a26a19 (0x7428 bytes) 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:3464 punch from 0x7a5c7b6 to 0x7a6a75d, (0xdfa7 bytes) 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:3465 writesame 0x1ebbef5 thru 0x1ebf629 data_size 0x7(0x3735 bytes) 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:3466 write 0x21062da thru 0x210dd17 (0x7a3e bytes) 2024-09-16T14:08:23.690 INFO:teuthology.orchestra.run.smithi079.stdout:3467 write 0x5d15fac thru 0x5d23fd8 (0xe02d bytes) 2024-09-16T14:08:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:3468 read 0x497a643 thru 0x49849fa (0xa3b8 bytes) 2024-09-16T14:08:23.696 INFO:teuthology.orchestra.run.smithi079.stdout:3470 read 0x9db2530 thru 0x9db2b91 (0x662 bytes) 2024-09-16T14:08:23.708 INFO:teuthology.orchestra.run.smithi079.stdout:3471 write 0xc9b74f4 thru 0xc9c57ce (0xe2db bytes) 2024-09-16T14:08:23.717 INFO:teuthology.orchestra.run.smithi079.stdout:3472 read 0x34456be thru 0x344766d (0x1fb0 bytes) 2024-09-16T14:08:23.717 INFO:teuthology.orchestra.run.smithi079.stdout:3473 read 0x78a192e thru 0x78ae8cc (0xcf9f bytes) 2024-09-16T14:08:23.717 INFO:teuthology.orchestra.run.smithi079.stdout:3475 write 0x949838 thru 0x95123f (0x7a08 bytes) 2024-09-16T14:08:23.726 INFO:teuthology.orchestra.run.smithi079.stdout:3476 read 0x150432c thru 0x150eaae (0xa783 bytes) 2024-09-16T14:08:23.744 INFO:teuthology.orchestra.run.smithi079.stdout:3478 writesame 0xeb96ec4 thru 0xeb9ab0b data_size 0xc(0x3c48 bytes) 2024-09-16T14:08:23.744 INFO:teuthology.orchestra.run.smithi079.stdout:3479 read 0x2526718 thru 0x252f510 (0x8df9 bytes) 2024-09-16T14:08:23.744 INFO:teuthology.orchestra.run.smithi079.stdout:3481 punch from 0x8b5f953 to 0x8b67bfe, (0x82ab bytes) 2024-09-16T14:08:23.744 INFO:teuthology.orchestra.run.smithi079.stdout:3482 write 0x81b1f91 thru 0x81b5890 (0x3900 bytes) 2024-09-16T14:08:23.751 INFO:teuthology.orchestra.run.smithi079.stdout:3484 read 0xd27ed2f thru 0xd2821e7 (0x34b9 bytes) 2024-09-16T14:08:23.751 INFO:teuthology.orchestra.run.smithi079.stdout:3486 write 0x7a86848 thru 0x7a8c32d (0x5ae6 bytes) 2024-09-16T14:08:23.757 INFO:teuthology.orchestra.run.smithi079.stdout:3487 read 0xdffb8c1 thru 0xe0097bc (0xdefc bytes) 2024-09-16T14:08:23.758 INFO:teuthology.orchestra.run.smithi079.stdout:3490 write 0x40d726 thru 0x417980 (0xa25b bytes) 2024-09-16T14:08:23.761 INFO:teuthology.orchestra.run.smithi079.stdout:3493 read 0xccd1502 thru 0xccda814 (0x9313 bytes) 2024-09-16T14:08:23.762 INFO:teuthology.orchestra.run.smithi079.stdout:3494 write 0x5c45e7f thru 0x5c50228 (0xa3aa bytes) 2024-09-16T14:08:23.775 INFO:teuthology.orchestra.run.smithi079.stdout:3495 trunc from 0xeb9ab0c to 0x9958c0f 2024-09-16T14:08:23.788 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:23.785+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:23.796 INFO:teuthology.orchestra.run.smithi079.stdout:3497 read 0x22037c1 thru 0x22101f6 (0xca36 bytes) 2024-09-16T14:08:23.797 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.797 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.797 INFO:teuthology.orchestra.run.smithi079.stdout:3500 read 0x854507a thru 0x8553d81 (0xed08 bytes) 2024-09-16T14:08:23.811 INFO:teuthology.orchestra.run.smithi079.stdout:3501 trunc from 0x9958c0f to 0x21778a2 2024-09-16T14:08:23.838 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:23.838 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:23.838 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:23.838 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:23.840 INFO:teuthology.orchestra.run.smithi079.stdout:3503 trunc from 0x21778a2 to 0x6ca66c9 2024-09-16T14:08:23.848 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:23.851 INFO:teuthology.orchestra.run.smithi079.stdout:3504 trunc from 0x6ca66c9 to 0x809f1d4 2024-09-16T14:08:23.861 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.861 INFO:teuthology.orchestra.run.smithi079.stdout:3506 read 0x66ac681 thru 0x66ad0c0 (0xa40 bytes) 2024-09-16T14:08:23.861 INFO:teuthology.orchestra.run.smithi079.stdout:3507 read 0x42b6bdb thru 0x42c0d18 (0xa13e bytes) 2024-09-16T14:08:23.862 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:23.862 INFO:teuthology.orchestra.run.smithi079.stdout:3508 punch from 0x6d0f966 to 0x6d19308, (0x99a2 bytes) 2024-09-16T14:08:23.862 INFO:teuthology.orchestra.run.smithi079.stdout:3509 writesame 0x5d0582d thru 0x5d10068 data_size 0x25(0xa83c bytes) 2024-09-16T14:08:23.868 INFO:teuthology.orchestra.run.smithi079.stdout:3510 write 0x1943ffc thru 0x1953218 (0xf21d bytes) 2024-09-16T14:08:23.872 INFO:teuthology.orchestra.run.smithi079.stdout:3511 read 0x5a1f425 thru 0x5a2ceef (0xdacb bytes) 2024-09-16T14:08:23.873 INFO:teuthology.orchestra.run.smithi079.stdout:3513 write 0x14890a5 thru 0x148b1fb (0x2157 bytes) 2024-09-16T14:08:23.878 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.878 INFO:teuthology.orchestra.run.smithi079.stdout:3517 read 0x4528fcd thru 0x452f096 (0x60ca bytes) 2024-09-16T14:08:23.878 INFO:teuthology.orchestra.run.smithi079.stdout:3518 write 0x486cf08 thru 0x4876221 (0x931a bytes) 2024-09-16T14:08:23.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:23.889 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:23.885+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:23.902 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:23.907 INFO:teuthology.orchestra.run.smithi079.stdout:3519 write 0xceded47 thru 0xcee50ea (0x63a4 bytes) 2024-09-16T14:08:23.924 INFO:teuthology.orchestra.run.smithi079.stdout:3520 trunc from 0xcee50eb to 0x9b5efff 2024-09-16T14:08:23.924 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:23.941 INFO:teuthology.orchestra.run.smithi079.stdout:3523 read 0x9934716 thru 0x9942a85 (0xe370 bytes) 2024-09-16T14:08:23.946 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:23.949 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:23.949 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:23.961 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:23.962 INFO:teuthology.orchestra.run.smithi079.stdout:3524 writesame 0xee024a0 thru 0xee04a11 data_size 0x1(0x2572 bytes) 2024-09-16T14:08:23.962 INFO:teuthology.orchestra.run.smithi079.stdout:3528 read 0x943b56e thru 0x944524f (0x9ce2 bytes) 2024-09-16T14:08:23.962 INFO:teuthology.orchestra.run.smithi079.stdout:3530 read 0x9eadae1 thru 0x9eb8187 (0xa6a7 bytes) 2024-09-16T14:08:23.962 INFO:teuthology.orchestra.run.smithi079.stdout:3532 writesame 0xc9bc43b thru 0xc9be99a data_size 0x20(0x2560 bytes) 2024-09-16T14:08:23.967 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:23.967 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:23.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:23.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:23.968 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:23.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:23.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:23.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:23.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:23.969 INFO:teuthology.orchestra.run.smithi079.stdout:3534 read 0xea9406a thru 0xea9c15b (0x80f2 bytes) 2024-09-16T14:08:23.969 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.969 INFO:teuthology.orchestra.run.smithi079.stdout:3538 write 0x29fc6ec thru 0x2a0a6a9 (0xdfbe bytes) 2024-09-16T14:08:23.971 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:23.971 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:23.977 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:23.977 INFO:teuthology.orchestra.run.smithi079.stdout:3541 write 0x6038b63 thru 0x6047eb3 (0xf351 bytes) 2024-09-16T14:08:23.983 INFO:teuthology.orchestra.run.smithi079.stdout:3542 read 0x6da946 thru 0x6de23c (0x38f7 bytes) 2024-09-16T14:08:23.984 INFO:teuthology.orchestra.run.smithi079.stdout:3543 write 0xa9cc5e2 thru 0xa9d60c4 (0x9ae3 bytes) 2024-09-16T14:08:23.990 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:23.989+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:23.993 INFO:teuthology.orchestra.run.smithi079.stdout:3544 punch from 0x7a42ce9 to 0x7a4840c, (0x5723 bytes) 2024-09-16T14:08:23.993 INFO:teuthology.orchestra.run.smithi079.stdout:3548 writesame 0x51e2dd6 thru 0x51f287d data_size 0x34(0xfaa8 bytes) 2024-09-16T14:08:24.001 INFO:teuthology.orchestra.run.smithi079.stdout:3551 punch from 0x95abc7e to 0x95b0214, (0x4596 bytes) 2024-09-16T14:08:24.001 INFO:teuthology.orchestra.run.smithi079.stdout:3552 write 0x5772beb thru 0x57754fc (0x2912 bytes) 2024-09-16T14:08:24.004 INFO:teuthology.orchestra.run.smithi079.stdout:3553 write 0xe8c3063 thru 0xe8cc94a (0x98e8 bytes) 2024-09-16T14:08:24.034 INFO:teuthology.orchestra.run.smithi079.stdout:3555 trunc from 0xee04a12 to 0x23b5c8d 2024-09-16T14:08:24.062 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.062 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.067 INFO:teuthology.orchestra.run.smithi079.stdout:3557 writesame 0x2e4d850 thru 0x2e5d240 data_size 0x5(0xf9f1 bytes) 2024-09-16T14:08:24.067 INFO:teuthology.orchestra.run.smithi079.stdout:3558 writesame 0x24bc728 thru 0x24bf329 data_size 0x2b(0x2c02 bytes) 2024-09-16T14:08:24.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:24.074 INFO:teuthology.orchestra.run.smithi079.stdout:3559 punch from 0xc2f212 to 0xc3903d, (0x9e2b bytes) 2024-09-16T14:08:24.074 INFO:teuthology.orchestra.run.smithi079.stdout:3561 punch from 0x2d160c1 to 0x2d1fd35, (0x9c74 bytes) 2024-09-16T14:08:24.074 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:24.075 INFO:teuthology.orchestra.run.smithi079.stdout:3563 read 0x1c1ff95 thru 0x1c29333 (0x939f bytes) 2024-09-16T14:08:24.075 INFO:teuthology.orchestra.run.smithi079.stdout:3564 write 0x2d5fdea thru 0x2d627f2 (0x2a09 bytes) 2024-09-16T14:08:24.084 INFO:teuthology.orchestra.run.smithi079.stdout:3565 punch from 0x28b4d54 to 0x28b7259, (0x2505 bytes) 2024-09-16T14:08:24.084 INFO:teuthology.orchestra.run.smithi079.stdout:3566 write 0x1b004c1 thru 0x1b0e137 (0xdc77 bytes) 2024-09-16T14:08:24.086 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:24.086 INFO:teuthology.orchestra.run.smithi079.stdout:3568 punch from 0xacf123 to 0xada4b9, (0xb396 bytes) 2024-09-16T14:08:24.090 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:24.086+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:24.102 INFO:teuthology.orchestra.run.smithi079.stdout:3569 trunc from 0x2e5d241 to 0xa39311b 2024-09-16T14:08:24.113 INFO:teuthology.orchestra.run.smithi079.stdout:3570 read 0x1ae6395 thru 0x1af58c5 (0xf531 bytes) 2024-09-16T14:08:24.114 INFO:teuthology.orchestra.run.smithi079.stdout:3571 read 0x9a5b47b thru 0x9a6ab31 (0xf6b7 bytes) 2024-09-16T14:08:24.115 INFO:teuthology.orchestra.run.smithi079.stdout:3572 trunc from 0xa39311b to 0xb0f9425 2024-09-16T14:08:24.127 INFO:teuthology.orchestra.run.smithi079.stdout:3573 read 0x84eac2e thru 0x84f9ef7 (0xf2ca bytes) 2024-09-16T14:08:24.148 INFO:teuthology.orchestra.run.smithi079.stdout:3576 writesame 0xdf6bc1b thru 0xdf73df0 data_size 0x1(0x81d6 bytes) 2024-09-16T14:08:24.148 INFO:teuthology.orchestra.run.smithi079.stdout:3578 write 0x28c9ce3 thru 0x28ca188 (0x4a6 bytes) 2024-09-16T14:08:24.150 INFO:teuthology.orchestra.run.smithi079.stdout:3580 clone 31 order 22 su 524288 sc 2 2024-09-16T14:08:24.171 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.171 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.182 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:24.223 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:24.218+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:24.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:08:24.292 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:24.286+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.322 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.323 INFO:teuthology.orchestra.run.smithi079.stdout:leaving image image_client.0-clone30 intact 2024-09-16T14:08:24.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:24.393 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:24.389+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:24.428 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.428 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.437 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:24.494 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:24.493+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:24.539 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.539 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.555 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:24.566 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-16T14:08:24.608 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:24.593+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:24.654 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.654 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.664 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:24.696 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:24.693+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:24.759 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.759 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:24.796 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:24.790+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:24.869 INFO:teuthology.orchestra.run.smithi079.stdout:3583 writesame 0xb69ce2e thru 0xb6ab9d8 data_size 0x1(0xebab bytes) 2024-09-16T14:08:24.874 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.875 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.887 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:24.897 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:24.893+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:24.923 INFO:teuthology.orchestra.run.smithi079.stdout:3584 write 0xc1be66 thru 0xc21f7c (0x6117 bytes) 2024-09-16T14:08:24.933 INFO:teuthology.orchestra.run.smithi079.stdout:3586 write 0x95bced6 thru 0x95c6322 (0x944d bytes) 2024-09-16T14:08:24.958 INFO:teuthology.orchestra.run.smithi079.stdout:3587 trunc from 0xdf73df1 to 0x1dd66ce 2024-09-16T14:08:24.984 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:24.984 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:24.990 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:24.990 INFO:teuthology.orchestra.run.smithi079.stdout:3589 punch from 0x8dc9e1 to 0x8e2132, (0x5751 bytes) 2024-09-16T14:08:24.990 INFO:teuthology.orchestra.run.smithi079.stdout:3592 read 0xb4d019 thru 0xb5c4b2 (0xf49a bytes) 2024-09-16T14:08:24.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:24.998 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:24.990+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:25.001 INFO:teuthology.orchestra.run.smithi079.stdout:3593 writesame 0x1faee61 thru 0x1fbebf7 data_size 0x1(0xfd97 bytes) 2024-09-16T14:08:25.019 INFO:teuthology.orchestra.run.smithi079.stdout:3595 writesame 0x4bdb53c thru 0x4be0658 data_size 0x1(0x511d bytes) 2024-09-16T14:08:25.031 INFO:teuthology.orchestra.run.smithi079.stdout:3596 write 0x51e6c96 thru 0x51f2091 (0xb3fc bytes) 2024-09-16T14:08:25.045 INFO:teuthology.orchestra.run.smithi079.stdout:3597 trunc from 0x51f2092 to 0x15adf75 2024-09-16T14:08:25.099 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:25.094+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:25.105 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:25.106 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:25.110 INFO:teuthology.orchestra.run.smithi079.stdout:3598 writesame 0xca5c803 thru 0xca5dcbc data_size 0x7(0x14ba bytes) 2024-09-16T14:08:25.110 INFO:teuthology.orchestra.run.smithi079.stdout:3599 writesame 0x4d3666b thru 0x4d384d3 data_size 0xf(0x1e69 bytes) 2024-09-16T14:08:25.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:25.116 INFO:teuthology.orchestra.run.smithi079.stdout:3600 writesame 0x1e115e6 thru 0x1e20f2e data_size 0x1(0xf949 bytes) 2024-09-16T14:08:25.168 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.168 INFO:teuthology.orchestra.run.smithi079.stdout:3607 punch from 0x9478fc3 to 0x947a632, (0x166f bytes) 2024-09-16T14:08:25.168 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.168 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.168 INFO:teuthology.orchestra.run.smithi079.stdout:3614 write 0x402df53 thru 0x403a6b5 (0xc763 bytes) 2024-09-16T14:08:25.173 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.173 INFO:teuthology.orchestra.run.smithi079.stdout:3616 write 0x5b335bc thru 0x5b33b58 (0x59d bytes) 2024-09-16T14:08:25.183 INFO:teuthology.orchestra.run.smithi079.stdout:3617 read 0x6f300ca thru 0x6f3a4ce (0xa405 bytes) 2024-09-16T14:08:25.185 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:08:25.186 INFO:teuthology.orchestra.run.smithi079.stdout:3618 trunc from 0xca5dcbd to 0xb44252f 2024-09-16T14:08:25.199 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:25.194+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:25.200 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.200 INFO:teuthology.orchestra.run.smithi079.stdout:3622 punch from 0x8bde89b to 0x8bea31a, (0xba7f bytes) 2024-09-16T14:08:25.209 INFO:teuthology.orchestra.run.smithi079.stdout:3623 trunc from 0xb44252f to 0x600f256 2024-09-16T14:08:25.219 INFO:teuthology.orchestra.run.smithi079.stdout:3624 punch from 0x4118d36 to 0x4125960, (0xcc2a bytes) 2024-09-16T14:08:25.219 INFO:teuthology.orchestra.run.smithi079.stdout:3625 read 0x18d214 thru 0x195afb (0x88e8 bytes) 2024-09-16T14:08:25.221 INFO:teuthology.orchestra.run.smithi079.stdout:3626 read 0x1deeee7 thru 0x1df1fb4 (0x30ce bytes) 2024-09-16T14:08:25.221 INFO:teuthology.orchestra.run.smithi079.stdout:3627 read 0x3a6426c thru 0x3a68402 (0x4197 bytes) 2024-09-16T14:08:25.221 INFO:teuthology.orchestra.run.smithi079.stdout:3628 punch from 0xc3bd4c to 0xc4a987, (0xec3b bytes) 2024-09-16T14:08:25.227 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:25.227 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:25.229 INFO:teuthology.orchestra.run.smithi079.stdout:3629 trunc from 0x600f256 to 0x1ef8b99 2024-09-16T14:08:25.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:25.263 INFO:teuthology.orchestra.run.smithi079.stdout:3631 trunc from 0x1ef8b99 to 0x4aafc39 2024-09-16T14:08:25.275 INFO:teuthology.orchestra.run.smithi079.stdout:3632 read 0x3af9310 thru 0x3af97ae (0x49f bytes) 2024-09-16T14:08:25.301 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:25.297+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:25.305 INFO:teuthology.orchestra.run.smithi079.stdout:3633 write 0xdf2e70c thru 0xdf3beae (0xd7a3 bytes) 2024-09-16T14:08:25.313 INFO:teuthology.orchestra.run.smithi079.stdout:3634 write 0x892e3b0 thru 0x8934d80 (0x69d1 bytes) 2024-09-16T14:08:25.319 INFO:teuthology.orchestra.run.smithi079.stdout:3636 punch from 0x38a467e to 0x38b2c56, (0xe5d8 bytes) 2024-09-16T14:08:25.319 INFO:teuthology.orchestra.run.smithi079.stdout:3638 read 0x736cb75 thru 0x737008a (0x3516 bytes) 2024-09-16T14:08:25.319 INFO:teuthology.orchestra.run.smithi079.stdout:3639 write 0x287c181 thru 0x288b448 (0xf2c8 bytes) 2024-09-16T14:08:25.324 INFO:teuthology.orchestra.run.smithi079.stdout:3640 write 0x76c5572 thru 0x76d3eba (0xe949 bytes) 2024-09-16T14:08:25.327 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.327 INFO:teuthology.orchestra.run.smithi079.stdout:3645 read 0x16654b thru 0x168c4b (0x2701 bytes) 2024-09-16T14:08:25.329 INFO:teuthology.orchestra.run.smithi079.stdout:3648 trunc from 0xdf3beaf to 0xd5713e0 2024-09-16T14:08:25.348 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:25.348 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:25.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:25.358 INFO:teuthology.orchestra.run.smithi079.stdout:3649 write 0x2c5e90a thru 0x2c624ff (0x3bf6 bytes) 2024-09-16T14:08:25.361 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.373 INFO:teuthology.orchestra.run.smithi079.stdout:3654 write 0xdb06852 thru 0xdb14d29 (0xe4d8 bytes) 2024-09-16T14:08:25.378 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.378 INFO:teuthology.orchestra.run.smithi079.stdout:3658 write 0xc360e0 thru 0xc3bdc0 (0x5ce1 bytes) 2024-09-16T14:08:25.381 INFO:teuthology.orchestra.run.smithi079.stdout:3659 read 0x52f9ef2 thru 0x52fdcec (0x3dfb bytes) 2024-09-16T14:08:25.381 INFO:teuthology.orchestra.run.smithi079.stdout:3660 write 0x8da2ea7 thru 0x8db2640 (0xf79a bytes) 2024-09-16T14:08:25.387 INFO:teuthology.orchestra.run.smithi079.stdout:3661 write 0xadb5557 thru 0xadbe136 (0x8be0 bytes) 2024-09-16T14:08:25.393 INFO:teuthology.orchestra.run.smithi079.stdout:3662 punch from 0x20bb859 to 0x20bd838, (0x1fdf bytes) 2024-09-16T14:08:25.402 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:25.401+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:25.404 INFO:teuthology.orchestra.run.smithi079.stdout:3665 trunc from 0xdb14d2a to 0x7efb45e 2024-09-16T14:08:25.422 INFO:teuthology.orchestra.run.smithi079.stdout:3666 read 0x4cfec2c thru 0x4d079aa (0x8d7f bytes) 2024-09-16T14:08:25.422 INFO:teuthology.orchestra.run.smithi079.stdout:3668 punch from 0x200b7c to 0x201258, (0x6dc bytes) 2024-09-16T14:08:25.422 INFO:teuthology.orchestra.run.smithi079.stdout:3669 read 0xc26e2d thru 0xc32f39 (0xc10d bytes) 2024-09-16T14:08:25.423 INFO:teuthology.orchestra.run.smithi079.stdout:3672 write 0xe6df48 thru 0xe77e2d (0x9ee6 bytes) 2024-09-16T14:08:25.432 INFO:teuthology.orchestra.run.smithi079.stdout:3673 trunc from 0x7efb45e to 0x55d3aad 2024-09-16T14:08:25.465 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:25.465 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:25.467 INFO:teuthology.orchestra.run.smithi079.stdout:3674 write 0x88fd505 thru 0x8905a16 (0x8512 bytes) 2024-09-16T14:08:25.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:25.499 INFO:teuthology.orchestra.run.smithi079.stdout:3675 write 0xdb6e738 thru 0xdb7441d (0x5ce6 bytes) 2024-09-16T14:08:25.505 INFO:teuthology.orchestra.run.smithi079.stdout:3676 write 0xb12e480 thru 0xb134ee9 (0x6a6a bytes) 2024-09-16T14:08:25.505 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:25.501+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:25.512 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.512 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.512 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:25.537 INFO:teuthology.orchestra.run.smithi079.stdout:3680 trunc from 0xdb7441e to 0xc1eb76 2024-09-16T14:08:25.586 INFO:teuthology.orchestra.run.smithi079.stdout:3681 write 0x4a948fe thru 0x4a955d3 (0xcd6 bytes) 2024-09-16T14:08:25.592 INFO:teuthology.orchestra.run.smithi079.stdout:3682 punch from 0x48fe154 to 0x49034ab, (0x5357 bytes) 2024-09-16T14:08:25.592 INFO:teuthology.orchestra.run.smithi079.stdout:3683 punch from 0x1589623 to 0x158f9da, (0x63b7 bytes) 2024-09-16T14:08:25.592 INFO:teuthology.orchestra.run.smithi079.stdout:3684 read 0x3b209b5 thru 0x3b27325 (0x6971 bytes) 2024-09-16T14:08:25.592 INFO:teuthology.orchestra.run.smithi079.stdout:3685 punch from 0x38b8598 to 0x38c81e4, (0xfc4c bytes) 2024-09-16T14:08:25.604 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:25.598+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:25.610 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:25.611 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:25.611 INFO:teuthology.orchestra.run.smithi079.stdout:3686 write 0x901d657 thru 0x9021c90 (0x463a bytes) 2024-09-16T14:08:25.616 INFO:teuthology.orchestra.run.smithi079.stdout:3690 writesame 0x43edc3a thru 0x43fc2fc data_size 0x55(0xe6c3 bytes) 2024-09-16T14:08:25.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:25.638 INFO:teuthology.orchestra.run.smithi079.stdout:3692 write 0xc74a3f5 thru 0xc7508a3 (0x64af bytes) 2024-09-16T14:08:25.645 INFO:teuthology.orchestra.run.smithi079.stdout:3693 clone 32 order 24 su 524288 sc 10 2024-09-16T14:08:25.705 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:25.702+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:25.705 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:25.701+0000 7f3f1a52d540 -1 osd.3 58 log_to_monitors true 2024-09-16T14:08:25.746 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:25.747 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:25.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:25.806 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:25.805+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:25.900 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:25.900 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:25.907 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:25.905+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:25.909 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:25.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:26.008 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:26.002+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:26.018 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:26.018 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:26.027 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:26.109 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:26.105+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:26.114 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:26.131 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:26.131 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:26.138 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:26.140 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:26.157 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:26.180 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:26.210 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:26.202+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:26.219 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:26.217+0000 7f3f0ea70700 -1 osd.3 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:08:26.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:26.247 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:26.247 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:26.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:26.247 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:26.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.247 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:26.247 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.906048795000004, 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:26.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:26.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.906026397999995, 2024-09-16T14:08:26.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:26.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.905419687000006, 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:26.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:26.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:26.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:26.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:26.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:26.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.905345359999998, 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:26.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:26.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 58.934420568, 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:26.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:26.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.306461986999999, 2024-09-16T14:08:26.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:26.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:26.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.284 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.776301521000001, 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:26.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:26.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.128752989999999, 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101781 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582507196999998, 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019058275, 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101781 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:08:26.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660443+0000", 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660450+0000", 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999997e-06 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660454+0000", 2024-09-16T14:08:26.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0029999999999996e-06 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679188+0000", 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018734869000000001 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679271+0000", 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2721999999999998e-05 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679499+0000", 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227612 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101783 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582490019000002, 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019291800000000001, 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101783 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4499999999999997e-07 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660465+0000", 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4659999999999996e-06 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660471+0000", 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5250000000000002e-06 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660480+0000", 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5779999999999999e-06 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679530+0000", 2024-09-16T14:08:26.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019050085000000001 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679633+0000", 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103341 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679750+0000", 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011636 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582462750000001, 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582445559, 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:26.309 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:26.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:26.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582408542, 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:26.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582387866000001, 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:26.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582345686, 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:26.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:26.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582311461, 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:26.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:26.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582281745, 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:26.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:26.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.582262083, 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:26.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:26.344 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.573501958000001, 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:26.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:26.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4887754160000002, 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:08:26.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.353 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:26.355 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:26.355 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:26.355 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:26.355 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:26.356 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:26.356 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone31 from 0xc7508a4 (overlap 0xc1eb76) to 0x0 2024-09-16T14:08:26.357 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:26.353+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:26.407 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:26.426 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:26.448 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:26.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:26.449 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:26.449 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:26.449 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-16T14:08:26.449 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:26.449 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:26.450 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:26.450 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:26.453 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:08:26.454 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:08:26.455 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:26.450+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:26.460 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:26.460 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:26.460 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:26.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:26.512 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-16T14:08:26.556 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:26.550+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:26.572 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:26.572 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:26.581 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:26.657 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:26.653+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8982267300000002, 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:08:26.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:08:26.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:08:26.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8982103910000001, 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:08:26.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:08:26.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8981852259999998, 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:08:26.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:08:26.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:08:26.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:08:26.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:08:26.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8972812589999997, 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:26.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:08:26.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8972719199999997, 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8969817979999997, 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:08:26.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:08:26.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:08:26.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8967631039999997, 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:08:26.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:08:26.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8903695249999997, 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:08:26.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8864579619999997, 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:08:26.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:08:26.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8864267569999997, 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:08:26.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8856073169999998, 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:08:26.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:08:26.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:08:26.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.796 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8855625079999996, 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:08:26.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:08:26.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8825532540000003, 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:08:26.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:08:26.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8825362940000003, 2024-09-16T14:08:26.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:08:26.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8825247980000004, 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:08:26.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:08:26.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8777778879999998, 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:26.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:08:26.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8701452569999999, 2024-09-16T14:08:26.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:08:26.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8701330499999997, 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:26.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:26.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:08:26.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8678781349999998, 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:08:26.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:08:26.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.839 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8676538840000001, 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.840 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:08:26.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:26.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:08:26.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:08:26.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.845 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.845 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.845 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:26.846 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:26.847 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:26.838+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:26.929 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:26.929 INFO:teuthology.orchestra.run.smithi079.stdout:3696 write 0xbc06cf9 thru 0xbc06d16 (0x1e bytes) 2024-09-16T14:08:26.945 INFO:teuthology.orchestra.run.smithi079.stdout:3697 punch from 0xb5f3a41 to 0xb5f3f14, (0x4d3 bytes) 2024-09-16T14:08:26.946 INFO:teuthology.orchestra.run.smithi079.stdout:3698 read 0x90f7e18 thru 0x9101a96 (0x9c7f bytes) 2024-09-16T14:08:26.946 INFO:teuthology.orchestra.run.smithi079.stdout:3699 read 0x1c5c4bf thru 0x1c6c1d7 (0xfd19 bytes) 2024-09-16T14:08:26.947 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:26.942+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:26.947 INFO:teuthology.orchestra.run.smithi079.stdout:3700 read 0xb1b5ac thru 0xb1f4eb (0x3f40 bytes) 2024-09-16T14:08:26.959 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:26.959 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:26.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.959 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:145201 3.12 3:4e075e66:::rbd_data.113673b9891.0000000000000006:head [call rbd.sparse_copyup in=24b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:26.955318+0000", 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0023611470000000001, 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024310820000000002, 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 145201 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:26.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.955318+0000", 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.955318+0000", 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.955321+0000", 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.187e-06 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.955332+0000", 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1233000000000001e-05 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.955366+0000", 2024-09-16T14:08:26.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3979000000000001e-05 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.955377+0000", 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.149e-05 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.955467+0000", 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0008999999999998e-05 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.955718+0000", 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025076900000000002 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.957496+0000", 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001777736 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:26.957649+0000", 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153497 2024-09-16T14:08:26.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.965 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:26.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:26.965 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:08:26.965 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T14:08:26.965 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:26.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:26.973 INFO:teuthology.orchestra.run.smithi079.stdout:3702 write 0xe8e1f31 thru 0xe8e631d (0x43ed bytes) 2024-09-16T14:08:26.978 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:26.978 INFO:teuthology.orchestra.run.smithi079.stdout:3705 writesame 0xe403743 thru 0xe408f47 data_size 0x25(0x5805 bytes) 2024-09-16T14:08:26.981 INFO:teuthology.orchestra.run.smithi079.stdout:3706 writesame 0x387ef5a thru 0x387f4b8 data_size 0x5(0x55f bytes) 2024-09-16T14:08:27.004 INFO:teuthology.orchestra.run.smithi079.stdout:3707 trunc from 0xe8e631e to 0x468d372 2024-09-16T14:08:27.047 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:27.042+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:27.063 INFO:teuthology.orchestra.run.smithi079.stdout:3709 write 0x82d81e8 thru 0x82e5bb1 (0xd9ca bytes) 2024-09-16T14:08:27.072 INFO:teuthology.orchestra.run.smithi079.stdout:3710 trunc from 0x82e5bb2 to 0xb7b6852 2024-09-16T14:08:27.098 INFO:teuthology.orchestra.run.smithi079.stdout:3711 trunc from 0xb7b6852 to 0x30c8a1e 2024-09-16T14:08:27.116 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:27.116 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:27.116 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:27.116 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:27.116 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:27.125 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:27.145 INFO:teuthology.orchestra.run.smithi079.stdout:3712 write 0xdab88e7 thru 0xdab9c6f (0x1389 bytes) 2024-09-16T14:08:27.148 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:27.145+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:27.151 INFO:teuthology.orchestra.run.smithi079.stdout:3713 read 0x52ea7a thru 0x5360b9 (0x7640 bytes) 2024-09-16T14:08:27.152 INFO:teuthology.orchestra.run.smithi079.stdout:3715 read 0xaa9768c thru 0xaa9d204 (0x5b79 bytes) 2024-09-16T14:08:27.152 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:27.152 INFO:teuthology.orchestra.run.smithi079.stdout:3717 read 0x667ae64 thru 0x667c93f (0x1adc bytes) 2024-09-16T14:08:27.152 INFO:teuthology.orchestra.run.smithi079.stdout:3718 writesame 0xaa8054f thru 0xaa8d7ad data_size 0x1(0xd25f bytes) 2024-09-16T14:08:27.193 INFO:teuthology.orchestra.run.smithi079.stdout:3720 write 0x18e784d thru 0x18e8b41 (0x12f5 bytes) 2024-09-16T14:08:27.196 INFO:teuthology.orchestra.run.smithi079.stdout:3723 write 0xd41cb32 thru 0xd425842 (0x8d11 bytes) 2024-09-16T14:08:27.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:08:27.231 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.231 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.853216969999998, 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:08:27.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:08:27.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.849285552000001, 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.236 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:08:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.240 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.84926136, 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.241 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:08:27.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.245 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.845542088999998, 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:08:27.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.841625323999999, 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:08:27.250 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:08:27.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:08:27.254 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.837243045000001, 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.255 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:08:27.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:27.259 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.837219204, 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:27.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:08:27.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.837204016000001, 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:08:27.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.837189429999999, 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:08:27.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:27.273 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.837085779999999, 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:08:27.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.837058054, 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:27.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.834384438000001, 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:08:27.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.834357724, 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:08:27.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.834317556999999, 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.292 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:08:27.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.834227842000001, 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:27.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:08:27.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.833819038000001, 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:27.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.833718854000001, 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:08:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:08:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:08:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:08:27.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.310 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.832820216000002, 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:08:27.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:08:27.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.315 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.825621483999999, 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:27.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.823453191999999, 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:27.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:27.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:08:27.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:27.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.326 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:27.328 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:27.328 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:27.325+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:27.429 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:27.425+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:27.433 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:27.434 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:27.434 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:27.434 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:27.443 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:27.530 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:27.529+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:27.542 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:27.542 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:27.551 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:27.631 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:27.629+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.240665258000007, 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:08:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:08:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:08:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.402702935000001, 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:08:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:08:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:08:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8754639470000001, 2024-09-16T14:08:27.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:08:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.875440716, 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8754177690000002, 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:08:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8754012600000003, 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:08:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8753836340000003, 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:08:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8694678899999992, 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:08:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8694504849999998, 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:08:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8690961189999999, 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:08:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8659522600000003, 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:08:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:08:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:08:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8659369740000002, 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:08:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8659205649999997, 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:27.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8658453680000004, 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:08:27.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:08:27.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8657478209999994, 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.724 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:08:27.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.728 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8657225799999999, 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.729 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:08:27.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.733 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8656935259999994, 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:08:27.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:08:27.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8624295370000006, 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:08:27.738 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:08:27.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.742 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8624113359999992, 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.743 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:08:27.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:27.747 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8623721579999994, 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:27.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:08:27.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:08:27.752 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.752 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.752 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.752 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:27.752 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:27.752 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:27.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:27.754 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:27.746+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:27.854 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:27.846+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:27.864 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.62706845200000005, 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.62711078099999995, 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:08:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-16T14:08:27.870 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:27.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:27.956 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:27.950+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:27.978 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:27.979 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:27.979 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:27.979 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:27.979 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:27.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:28.057 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:28.050+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:28.084 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:28.084 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:28.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:28.084 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2024-09-16T14:08:28.084 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:28.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:28.158 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:28.150+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:28.197 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:28.197 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:28.197 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:28.197 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:28.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:28.225 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:08:28.258 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:28.257+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:28.310 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:28.310 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:28.311 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:28.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:28.362 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:28.361+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:28.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:28.461 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:28.457+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:28.476 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:28.476 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562806434000002, 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.477 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:28.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:28.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562796706999997, 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:28.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:28.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562773143999998, 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:28.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:28.492 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562745905, 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:28.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:28.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562736491999999, 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.498 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:28.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:28.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562712173000001, 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.503 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:28.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:28.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562692566999999, 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:28.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:28.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562672378999999, 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:28.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:28.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562649110000002, 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:28.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:28.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.523 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562618860000001, 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:28.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:28.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562602108, 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.529 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:28.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:28.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562584844, 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.534 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:28.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:28.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562546007999998, 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:28.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562511180000001, 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:28.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562475294000002, 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:28.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:28.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562452702000002, 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562423629999998, 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:28.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562405284, 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:28.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:28.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562387944999998, 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:28.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:28.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.562372021999998, 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:28.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:28.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:28.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:28.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:28.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:28.583 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:28.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:28.585 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:28.581+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:28.586 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:28.610 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:28.628 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:28.646 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:28.667 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:28.685 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:28.678+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:28.689 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:28.709 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:28.709 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:28.710 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:28.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:28.731 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-16T14:08:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:28.734 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:28.786 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:28.778+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:28.822 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:28.823 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:28.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:28.888 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:28.882+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:28.931 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:28.931 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:28.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:28.989 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:28.985+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:29.039 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:29.039 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:29.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:29.090 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:29.089+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:29.191 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:29.186+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:29.194 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:29.194 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:29.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:29.229 INFO:teuthology.orchestra.run.smithi079.stdout:3725 read 0x27fd1a6 thru 0x2805f06 (0x8d61 bytes) 2024-09-16T14:08:29.240 INFO:teuthology.orchestra.run.smithi079.stdout:3727 trunc from 0xdab9c70 to 0x9feb463 2024-09-16T14:08:29.278 INFO:teuthology.orchestra.run.smithi079.stdout:3728 write 0xcbae2dd thru 0xcbb471d (0x6441 bytes) 2024-09-16T14:08:29.285 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:29.285 INFO:teuthology.orchestra.run.smithi079.stdout:3730 punch from 0xb370cad to 0xb377855, (0x6ba8 bytes) 2024-09-16T14:08:29.285 INFO:teuthology.orchestra.run.smithi079.stdout:3733 read 0xa488bd7 thru 0xa491013 (0x843d bytes) 2024-09-16T14:08:29.285 INFO:teuthology.orchestra.run.smithi079.stdout:3734 write 0xb031c79 thru 0xb034b3c (0x2ec4 bytes) 2024-09-16T14:08:29.292 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:29.286+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:29.292 INFO:teuthology.orchestra.run.smithi079.stdout:3737 read 0xc9090a6 thru 0xc90caff (0x3a5a bytes) 2024-09-16T14:08:29.292 INFO:teuthology.orchestra.run.smithi079.stdout:3738 punch from 0x8a80cc1 to 0x8a82d6a, (0x20a9 bytes) 2024-09-16T14:08:29.293 INFO:teuthology.orchestra.run.smithi079.stdout:3739 write 0xc085016 thru 0xc0861ee (0x11d9 bytes) 2024-09-16T14:08:29.318 INFO:teuthology.orchestra.run.smithi079.stdout:3740 write 0xe913004 thru 0xe9134b3 (0x4b0 bytes) 2024-09-16T14:08:29.322 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:29.322 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:29.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:29.333 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:29.333 INFO:teuthology.orchestra.run.smithi079.stdout:3742 read 0x4085a28 thru 0x409404d (0xe626 bytes) 2024-09-16T14:08:29.353 INFO:teuthology.orchestra.run.smithi079.stdout:3744 trunc from 0xe9134b4 to 0x3bb5785 2024-09-16T14:08:29.387 INFO:teuthology.orchestra.run.smithi079.stdout:3745 read 0x2d1ad36 thru 0x2d20525 (0x57f0 bytes) 2024-09-16T14:08:29.393 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:29.386+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:29.409 INFO:teuthology.orchestra.run.smithi079.stdout:3748 write 0x862f81f thru 0x86383e6 (0x8bc8 bytes) 2024-09-16T14:08:29.428 INFO:teuthology.orchestra.run.smithi079.stdout:3749 write 0x9eec26e thru 0x9ef545c (0x91ef bytes) 2024-09-16T14:08:29.432 INFO:teuthology.orchestra.run.smithi079.stdout:3750 read 0x6567e5b thru 0x65724fb (0xa6a1 bytes) 2024-09-16T14:08:29.432 INFO:teuthology.orchestra.run.smithi079.stdout:3751 punch from 0x4b09145 to 0x4b13d51, (0xac0c bytes) 2024-09-16T14:08:29.432 INFO:teuthology.orchestra.run.smithi079.stdout:3752 write 0xc5815 thru 0xd3ee6 (0xe6d2 bytes) 2024-09-16T14:08:29.449 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:29.449 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:29.456 INFO:teuthology.orchestra.run.smithi079.stdout:3753 writesame 0xde5c5a0 thru 0xde6a08b data_size 0x2(0xdaec bytes) 2024-09-16T14:08:29.456 INFO:teuthology.orchestra.run.smithi079.stdout:3755 clone 33 order 20 su 32768 sc 8 2024-09-16T14:08:29.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:29.494 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:29.486+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:29.567 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:29.567 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:29.576 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:29.598 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:29.590+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:29.671 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:29.671 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:29.680 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:29.727 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:29.722+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:29.802 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:29.794+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:29.845 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:29.903 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:29.900+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:29.943 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:29.943 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:29.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:30.006 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:30.000+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:30.055 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:30.055 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:30.064 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:30.107 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:30.102+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:30.166 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:30.167 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:30.176 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:30.210 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:30.202+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:30.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:08:30.288 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:30.288 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:30.297 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:30.343 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:30.338+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:30.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:30.412 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:30.408+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:30.492 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone32 from 0xde6a08c (overlap 0x30c8a1e) to 0x2be7c4e 2024-09-16T14:08:30.512 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:30.506+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:30.516 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:30.516 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:30.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:30.613 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:30.608+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:30.672 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:30.672 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:30.681 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:30.714 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:30.706+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:30.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:30.784 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:30.785 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:30.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:30.835 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:30.830+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:30.905 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:30.905 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:30.905 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:30.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:30.916 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:30.912+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:30.923 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:30.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:30.959 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:30.981 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:31.002 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:31.014 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:31.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:31.014 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:31.014 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:31.014 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:31.017 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:31.012+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:31.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:31.024 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:31.045 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:31.047 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:31.047 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:31.047 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:31.047 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:31.047 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:31.047 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:31.048 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:31.048 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:31.118 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:31.116+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:31.122 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:31.122 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.8810209640000002, 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:08:31.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.127 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.1d 84/83 [PushOp(3:bab684d6:::rbd_data.113619b3ee73.0000000000000013:16, version: 61'265, data_included: [152576~5120], data_size: 5120, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:bab684d6:::rbd_data.113619b3ee73.0000000000000013:16@61'265, size: 157696, copy_subset: [0~157696], clone_subset: {}, snapset: 16=[]:{16=[16]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:157696, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.229521+0000", 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8880479590000001, 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013243790000000001, 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.229521+0000", 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.229521+0000", 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.229521+0000", 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9099999999999991e-07 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.229544+0000", 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2571000000000001e-05 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.229559+0000", 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4663e-05 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.229566+0000", 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.878e-06 2024-09-16T14:08:31.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.229650+0000", 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4400999999999994e-05 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.229666+0000", 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6317999999999999e-05 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.230845+0000", 2024-09-16T14:08:31.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001178557 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.1d 84/83 [PushOp(3:bc8ffc69:::rbd_data.1136f7114f14.0000000000000015:head, version: 62'283, data_included: [0~79872,86016~38912,131072~24576,157696~14336,178176~18432,204800~12288,223232~20480,249856~12288,270336~18432,294912~14336,315392~20480,342016~12288,360448~20480,387072~14336,407552~116736], data_size: 438272, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:bc8ffc69:::rbd_data.1136f7114f14.0000000000000015:head@62'283, size: 524288, copy_subset: [0~524288], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:524288, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.240625+0000", 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8769434110000001, 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030990520000000001, 2024-09-16T14:08:31.132 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.240625+0000", 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.240625+0000", 2024-09-16T14:08:31.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.240626+0000", 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0780000000000001e-06 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.240985+0000", 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035886499999999999 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.241005+0000", 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9624999999999999e-05 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.241015+0000", 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9010000000000006e-06 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.241101+0000", 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6483000000000005e-05 2024-09-16T14:08:31.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.241109+0000", 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8439999999999998e-06 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.243724+0000", 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026152559999999998 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.1d 84/83 [PushOp(3:bef6dbfd:::rbd_data.1136e112ce26.0000000000000003:head, version: 70'319, data_included: [0~163840,180224~163840,360448~163840,540672~229376,786432~163840,966656~163840], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:bef6dbfd:::rbd_data.1136e112ce26.0000000000000003:head@70'319, size: 1632692, copy_subset: [0~1632692], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1130496, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.278873+0000", 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8386955340000002, 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026825260000000002, 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.278873+0000", 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.278873+0000", 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.278873+0000", 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0699999999999997e-07 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.279316+0000", 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00044233500000000003 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.279333+0000", 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6977e-05 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.279340+0000", 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3760000000000002e-06 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.279433+0000", 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2689000000000007e-05 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.279445+0000", 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2327000000000001e-05 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.281555+0000", 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021103150000000002 2024-09-16T14:08:31.140 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.1d 84/83 [PushOp(3:bef6dbfd:::rbd_data.1136e112ce26.0000000000000003:head, version: 70'319, data_included: [1146880~163840,1327104~229376,1572864~59828], data_size: 453044, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:bef6dbfd:::rbd_data.1136e112ce26.0000000000000003:head@70'319, size: 1632692, copy_subset: [0~1632692], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1632692, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1130496, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.282210+0000", 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8353588009999999, 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017514130000000001, 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.282210+0000", 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.282210+0000", 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.282210+0000", 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6599999999999995e-07 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.282376+0000", 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016583799999999999 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.282391+0000", 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4820999999999999e-05 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.282398+0000", 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7240000000000003e-06 2024-09-16T14:08:31.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.282480+0000", 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2108999999999996e-05 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.282493+0000", 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3145e-05 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.283961+0000", 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014680100000000001 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.1d 84/83 [PushOp(3:bef6dbfd:::rbd_data.1136e112ce26.0000000000000003:1c, version: 70'318, data_included: [0~163840,180224~163840,360448~163840,540672~229376,786432~163840,966656~163840], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:bef6dbfd:::rbd_data.1136e112ce26.0000000000000003:1c@70'318, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1c=[]:{1c=[1c]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1130496, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.285244+0000", 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8323241640000001, 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023659010000000001, 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.285244+0000", 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.285244+0000", 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.285245+0000", 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6899999999999997e-07 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.285746+0000", 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00050118700000000005 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.285770+0000", 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4226000000000001e-05 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.285781+0000", 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0514e-05 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.285871+0000", 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0488999999999994e-05 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.285882+0000", 2024-09-16T14:08:31.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0453e-05 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.287610+0000", 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017283629999999999 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.1d 84/83 [PushOp(3:bef6dbfd:::rbd_data.1136e112ce26.0000000000000003:1c, version: 70'318, data_included: [1146880~163840,1327104~229376,1572864~163840,1753088~163840,1933312~163840], data_size: 884736, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:bef6dbfd:::rbd_data.1136e112ce26.0000000000000003:1c@70'318, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 1c=[]:{1c=[1c]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1130496, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.288416+0000", 2024-09-16T14:08:31.149 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8291528509999999, 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002772278, 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.288416+0000", 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.288416+0000", 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.288416+0000", 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5899999999999995e-07 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.288928+0000", 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051190800000000005 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.288950+0000", 2024-09-16T14:08:31.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1906e-05 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.288962+0000", 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2167999999999999e-05 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.289053+0000", 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0976000000000004e-05 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.289066+0000", 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2302e-05 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.291188+0000", 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021221590000000002 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.153 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.1d 84/83 [PushOp(3:b82acafb:::rbd_data.113679f86037.0000000000000000:head, version: 75'330, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:b82acafb:::rbd_data.113679f86037.0000000000000000:head@75'330, size: 1048576, copy_subset: [0~1048576], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.301941+0000", 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.8156272090000001, 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027701039999999998, 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.301941+0000", 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.301941+0000", 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.301942+0000", 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0640000000000001e-06 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.302763+0000", 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00082103700000000005 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.302793+0000", 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0136000000000001e-05 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.302810+0000", 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6013000000000001e-05 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.302948+0000", 2024-09-16T14:08:31.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138775 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.302957+0000", 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1859999999999998e-06 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.304711+0000", 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017538930000000001 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.1d 84/83 [PushOp(3:bafa5999:::rbd_data.11368b74a150.0000000000000004:head, version: 77'343, data_included: [0~708608,741376~12288,786432~8192], data_size: 729088, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:bafa5999:::rbd_data.11368b74a150.0000000000000004:head@77'343, size: 806912, copy_subset: [0~806912], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:806912, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.319296+0000", 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7982728369999998, 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002547667, 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.319296+0000", 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.319296+0000", 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.319297+0000", 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3600000000000002e-07 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.319593+0000", 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000296001 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.319609+0000", 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6388999999999998e-05 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.319618+0000", 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9619999999999999e-06 2024-09-16T14:08:31.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.319703+0000", 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5041000000000001e-05 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.319714+0000", 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1199999999999999e-05 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.321843+0000", 2024-09-16T14:08:31.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002129138 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b703f557:::rbd_data.113618577e70.000000000000000a:17, version: 62'151, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b703f557:::rbd_data.113618577e70.000000000000000a:17@62'151, size: 262144, copy_subset: [0~262144], clone_subset: {}, snapset: 17=[]:{17=[17]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:262144, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.346509+0000", 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.77105939, 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001324993, 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.346509+0000", 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.346509+0000", 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.346510+0000", 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0400000000000005e-07 2024-09-16T14:08:31.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.346689+0000", 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017928599999999999 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.346713+0000", 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3651000000000001e-05 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.346725+0000", 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2318e-05 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.346810+0000", 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5031999999999995e-05 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.346818+0000", 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0779999999999996e-06 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.347834+0000", 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001015724 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b68ed584:::rbd_data.1136f7114f14.0000000000000049:18, version: 63'162, data_included: [36864~6342,206848~6144], data_size: 12486, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b68ed584:::rbd_data.1136f7114f14.0000000000000049:18@63'162, size: 212992, copy_subset: [0~212992], clone_subset: {}, snapset: 18=[]:{18=[18]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:212992, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.354958+0000", 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7626107630000001, 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013509500000000001, 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.166 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.354958+0000", 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.354958+0000", 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.354959+0000", 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0529999999999999e-06 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.354989+0000", 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0429e-05 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.355022+0000", 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2672000000000002e-05 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.355039+0000", 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7382000000000001e-05 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.355151+0000", 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111632 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.355163+0000", 2024-09-16T14:08:31.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2466999999999999e-05 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.356309+0000", 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001145315 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b774ddc9:::rbd_data.1136f7114f14.000000000000007f:18, version: 63'168, data_included: [0~10240,45056~10240,92160~8192,137216~10240,182272~10240,229376~8192], data_size: 57344, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b774ddc9:::rbd_data.1136f7114f14.000000000000007f:18@63'168, size: 237568, copy_subset: [0~237568], clone_subset: {}, snapset: 18=[]:{18=[18]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:237568, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:31.170 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.754189711, 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042357389999999997, 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:31.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363380+0000", 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363464+0000", 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4358999999999996e-05 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363496+0000", 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1704e-05 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363510+0000", 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.428e-05 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363685+0000", 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017525599999999999 2024-09-16T14:08:31.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363709+0000", 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3768999999999999e-05 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.367614+0000", 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003905281 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b5a9deab:::rbd_data.1136f7114f14.000000000000010e:18, version: 63'170, data_included: [137216~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b5a9deab:::rbd_data.1136f7114f14.000000000000010e:18@63'170, size: 141312, copy_subset: [0~141312], clone_subset: {}, snapset: 18=[]:{18=[18]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:141312, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.372454+0000", 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7451148519999999, 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00166215, 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.372454+0000", 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.372454+0000", 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.372454+0000", 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2099999999999995e-07 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.372494+0000", 2024-09-16T14:08:31.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-05 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.372520+0000", 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5360000000000001e-05 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.372544+0000", 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4314000000000001e-05 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.372678+0000", 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134487 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.372692+0000", 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3244999999999999e-05 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.374116+0000", 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001424023 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b312bc81:::rbd_data.11368cf3b67c.000000000000000a:head, version: 69'190, data_included: [0~557056,589824~32768,655360~32768,737280~32768,802816~393216], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:b312bc81:::rbd_data.11368cf3b67c.000000000000000a:head@69'190, size: 1900544, copy_subset: [0~1900544], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1196032, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.389709+0000", 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7278593889999998, 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028812899999999999, 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.389709+0000", 2024-09-16T14:08:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.389709+0000", 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.389710+0000", 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.24e-06 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.390500+0000", 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00078919400000000001 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.390528+0000", 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8989999999999999e-05 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.390545+0000", 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.390660+0000", 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115728 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.390670+0000", 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4949999999999997e-06 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.392590+0000", 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001920382 2024-09-16T14:08:31.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b312bc81:::rbd_data.11368cf3b67c.000000000000000a:head, version: 69'190, data_included: [1196032~196608,1425408~32768,1671168~229376], data_size: 458752, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b312bc81:::rbd_data.11368cf3b67c.000000000000000a:head@69'190, size: 1900544, copy_subset: [0~1900544], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1900544, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1196032, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.393964+0000", 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7236047210000001, 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015985540000000001, 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.183 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.393964+0000", 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.393964+0000", 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.393965+0000", 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7060000000000001e-06 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.394341+0000", 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00037518000000000002 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.394378+0000", 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7153000000000002e-05 2024-09-16T14:08:31.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.394395+0000", 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6866000000000001e-05 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.394523+0000", 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012840700000000001 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.394531+0000", 2024-09-16T14:08:31.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8229999999999995e-06 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.395562+0000", 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001031419 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b312bc81:::rbd_data.11368cf3b67c.000000000000000a:1b, version: 69'189, data_included: [0~557056,589824~32768,655360~32768,737280~32768,802816~393216], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b312bc81:::rbd_data.11368cf3b67c.000000000000000a:1b@69'189, size: 1916928, copy_subset: [0~1916928], clone_subset: {}, snapset: 1b=[]:{1b=[1b]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1196032, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.397440+0000", 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7201287700000001, 2024-09-16T14:08:31.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003879895, 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.397440+0000", 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.397440+0000", 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.397441+0000", 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.531e-06 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.398571+0000", 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001130275 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.398608+0000", 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6856000000000001e-05 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.398626+0000", 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7603000000000001e-05 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.398770+0000", 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014385400000000001 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.398785+0000", 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5116999999999999e-05 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.401320+0000", 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025346589999999999 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b312bc81:::rbd_data.11368cf3b67c.000000000000000a:1b, version: 69'189, data_included: [1196032~196608,1425408~32768,1671168~245760], data_size: 475136, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:b312bc81:::rbd_data.11368cf3b67c.000000000000000a:1b@69'189, size: 1916928, copy_subset: [0~1916928], clone_subset: {}, snapset: 1b=[]:{1b=[1b]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1916928, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1196032, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T14:08:31.191 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.402771+0000", 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7147973790000002, 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024316609999999999, 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.402771+0000", 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.402771+0000", 2024-09-16T14:08:31.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.402772+0000", 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.288e-06 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.403147+0000", 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00037420800000000001 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.403178+0000", 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1260000000000002e-05 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.403200+0000", 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.213e-05 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.403340+0000", 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013995699999999999 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.403352+0000", 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1895999999999999e-05 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.405203+0000", 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018509220000000001 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.195 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b6ccda4c:::rbd_data.113679f86037.0000000000000024:head, version: 75'195, data_included: [1019342~29234], data_size: 29234, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:b6ccda4c:::rbd_data.113679f86037.0000000000000024:head@75'195, size: 1048576, copy_subset: [0~1048576], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.410315+0000", 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.7072537990000001, 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001498744, 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.410315+0000", 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.410315+0000", 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.410316+0000", 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1650000000000001e-06 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.410364+0000", 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8386000000000001e-05 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.410391+0000", 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7039e-05 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.410406+0000", 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5213e-05 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.410526+0000", 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119296 2024-09-16T14:08:31.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.410536+0000", 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7489999999999993e-06 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.411813+0000", 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012778959999999999 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b7c8281b:::rbd_data.11368b74a150.0000000000000001:head, version: 77'214, data_included: [0~712704,745472~8192,786432~12288], data_size: 733184, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:b7c8281b:::rbd_data.11368b74a150.0000000000000001:head@77'214, size: 806912, copy_subset: [0~806912], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:806912, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.435125+0000", 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.682443444, 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029121659999999999, 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.435125+0000", 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.435125+0000", 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.435126+0000", 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3510000000000001e-06 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.435509+0000", 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038272799999999999 2024-09-16T14:08:31.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.435541+0000", 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2109e-05 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.435557+0000", 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6021000000000001e-05 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.435728+0000", 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017036500000000001 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.435741+0000", 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3152999999999999e-05 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.438037+0000", 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0022964389999999999 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b32cd35c:::rbd_data.11368b74a150.0000000000000003:head, version: 77'215, data_included: [0~708608,741376~12288,786432~12288], data_size: 733184, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:b32cd35c:::rbd_data.11368b74a150.0000000000000003:head@77'215, size: 806912, copy_subset: [0~806912], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:806912, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.439587+0000", 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 2.6779811169999999, 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024872589999999999, 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.439587+0000", 2024-09-16T14:08:31.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.439587+0000", 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.439588+0000", 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.077e-06 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.440201+0000", 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00061255399999999996 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.440233+0000", 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1704e-05 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.440247+0000", 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4610000000000001e-05 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.440377+0000", 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012952300000000001 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.440384+0000", 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4270000000000002e-06 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.442075+0000", 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001690364 2024-09-16T14:08:31.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.208 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:31.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:31.220 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:31.214+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:31.321 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:31.318+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:31.354 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:31.354 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:31.354 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:31.354 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:31.357 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:08:31.357 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:08:31.357 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T14:08:31.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T14:08:31.368 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:08:31.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:31.422 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:31.414+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:31.476 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:31.476 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:31.476 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:31.476 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:31.476 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:31.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:31.523 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:31.520+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:31.581 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:31.581 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667622569999999, 2024-09-16T14:08:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:31.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.589 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667612843000001, 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:31.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.594 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667589280000001, 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:31.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:31.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.600 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667562040999996, 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:31.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:31.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:31.605 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667552628000003, 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.611 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667528308999998, 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:31.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:31.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:31.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667508703000003, 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:31.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:31.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:31.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:31.623 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667488515000002, 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:31.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:31.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:31.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667465245999999, 2024-09-16T14:08:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:31.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:31.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:31.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667434995999997, 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:31.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:31.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:31.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.647 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667418243999997, 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:31.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667400979999996, 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:31.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:31.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:31.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.659 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667362144000002, 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.660 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:31.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:31.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.665 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667327315999998, 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.666 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:31.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:31.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:31.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:31.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667291429999999, 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.672 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:31.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667268837999998, 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:31.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667239766000002, 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:31.684 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:31.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:31.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:31.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667221419999997, 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667204081000001, 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:31.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:31.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.702 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 37.667188158000002, 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.703 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:31.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.709 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:31.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 88, 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "a22b38a6-94db-470d-bf35-45d7605f3bb0", 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-16T14:06:40.573726+0000", 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-16T14:08:31.347834+0000", 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-16T14:08:27.195701+0000", 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-16T14:06:42.748450+0000", 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T14:08:31.712 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5799936, 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 6, 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-16T14:08:31.713 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 3, 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "quincy", 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:52.822826+0000", 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-16T14:08:31.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-16T14:08:31.715 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:08:31.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "18", 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:08:31.717 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:08:31.718 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:08:31.719 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-16T14:08:31.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:58.526935+0000", 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:08:31.721 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-16T14:08:31.722 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "22", 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:08:31.723 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 22, 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:08:31.724 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:08:31.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:08:31.726 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "pool_client.0", 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:07:08.375919+0000", 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:08:31.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-16T14:08:31.728 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "86", 2024-09-16T14:08:31.729 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 36, 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 86, 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:08:31.730 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:08:31.731 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:08:31.732 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.733 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a060b161-fd5a-4cfb-bac8-f2acc32d7955", 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 85, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:31.734 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.735 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.736 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:31.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.738 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/22196", 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/22196", 2024-09-16T14:08:31.739 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6824/22196", 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/22196", 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "f5ba4b39-ceb8-46b3-938e-57249ed84b87", 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:31.740 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 85, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.742 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:31.743 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.744 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:31.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/22195", 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/22195", 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/22195", 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/22195", 2024-09-16T14:08:31.746 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "8b657f92-66c9-46a3-a31f-0bd0be414e4e", 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:31.747 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 87, 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-16T14:08:31.748 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:31.750 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-16T14:08:31.752 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6825/22197", 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6827/22197", 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6831/22197", 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6829/22197", 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:31.753 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "15937bd7-8964-4140-9f86-b3093c318d1e", 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 15, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 58, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 83, 2024-09-16T14:08:31.754 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 86, 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 59, 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-16T14:08:31.756 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/35927", 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/35927", 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/35927", 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/35927", 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "714baf80-ade0-4e88-ad4e-5171add1fc6a", 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.242828369140625, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 14, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 85, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6808", 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6809", 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6810", 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6811", 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6814", 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6815", 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6812", 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6813", 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6809/22362", 2024-09-16T14:08:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6811/22362", 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6815/22362", 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6813/22362", 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e5a21b84-81ab-45da-a081-3b504f5c4f3d", 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 87, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6800", 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6801", 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6802", 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6803", 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6806", 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6807", 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6804", 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6805", 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6801/22363", 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6803/22363", 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6807/22363", 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6805/22363", 2024-09-16T14:08:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a009857f-f798-4154-9ff4-f524f60b6c61", 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 86, 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6824", 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6825", 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6826", 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6827", 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6830", 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6831", 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6828", 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6829", 2024-09-16T14:08:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6825/22364", 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6827/22364", 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6831/22364", 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6829/22364", 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e3506002-ba8b-4251-8ef1-136334574a6d", 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:08:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 34, 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6816", 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6817", 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6818", 2024-09-16T14:08:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6819", 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6822", 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6823", 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:08:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6820", 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6821", 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6817/22365", 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6819/22365", 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6823/22365", 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6821/22365", 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.104631+0000", 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987403+0000", 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.018134+0000", 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987645+0000", 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 59 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:08:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.721507+0000", 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.869796+0000", 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.206576+0000", 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:08:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.038098+0000", 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.7", 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 0, 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 3 2024-09-16T14:08:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-16T14:08:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-16T14:08:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-16T14:08:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-16T14:08:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-16T14:08:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-16T14:08:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-16T14:08:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-16T14:08:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:31.798 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:31.799 INFO:tasks.thrashosds.thrasher:j is {'epoch': 88, 'fsid': 'a22b38a6-94db-470d-bf35-45d7605f3bb0', 'created': '2024-09-16T14:06:40.573726+0000', 'modified': '2024-09-16T14:08:31.347834+0000', 'last_up_change': '2024-09-16T14:08:27.195701+0000', 'last_in_change': '2024-09-16T14:06:42.748450+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T14:06:52.822826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T14:06:58.526935+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T14:07:08.375919+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 86, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': 'a060b161-fd5a-4cfb-bac8-f2acc32d7955', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 22196}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 22196}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6824', 'nonce': 22196}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 22196}]}, 'public_addr': '172.21.15.79:6817/22196', 'cluster_addr': '172.21.15.79:6819/22196', 'heartbeat_back_addr': '172.21.15.79:6824/22196', 'heartbeat_front_addr': '172.21.15.79:6821/22196', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f5ba4b39-ceb8-46b3-938e-57249ed84b87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 22195}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 22195}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 22195}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 22195}]}, 'public_addr': '172.21.15.79:6809/22195', 'cluster_addr': '172.21.15.79:6811/22195', 'heartbeat_back_addr': '172.21.15.79:6815/22195', 'heartbeat_front_addr': '172.21.15.79:6813/22195', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8b657f92-66c9-46a3-a31f-0bd0be414e4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6823', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 22197}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 22197}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 22197}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 22197}]}, 'public_addr': '172.21.15.79:6825/22197', 'cluster_addr': '172.21.15.79:6827/22197', 'heartbeat_back_addr': '172.21.15.79:6831/22197', 'heartbeat_front_addr': '172.21.15.79:6829/22197', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '15937bd7-8964-4140-9f86-b3093c318d1e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 58, 'up_from': 83, 'up_thru': 86, 'down_at': 59, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 35927}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 35927}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 35927}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 35927}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 35927}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 35927}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 35927}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 35927}]}, 'public_addr': '172.21.15.79:6801/35927', 'cluster_addr': '172.21.15.79:6803/35927', 'heartbeat_back_addr': '172.21.15.79:6807/35927', 'heartbeat_front_addr': '172.21.15.79:6805/35927', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '714baf80-ade0-4e88-ad4e-5171add1fc6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.242828369140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 22362}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 22362}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 22362}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 22362}]}, 'public_addr': '172.21.15.164:6809/22362', 'cluster_addr': '172.21.15.164:6811/22362', 'heartbeat_back_addr': '172.21.15.164:6815/22362', 'heartbeat_front_addr': '172.21.15.164:6813/22362', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e5a21b84-81ab-45da-a081-3b504f5c4f3d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 22363}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 22363}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 22363}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 22363}]}, 'public_addr': '172.21.15.164:6801/22363', 'cluster_addr': '172.21.15.164:6803/22363', 'heartbeat_back_addr': '172.21.15.164:6807/22363', 'heartbeat_front_addr': '172.21.15.164:6805/22363', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a009857f-f798-4154-9ff4-f524f60b6c61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 22364}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 22364}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 22364}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 22364}]}, 'public_addr': '172.21.15.164:6825/22364', 'cluster_addr': '172.21.15.164:6827/22364', 'heartbeat_back_addr': '172.21.15.164:6831/22364', 'heartbeat_front_addr': '172.21.15.164:6829/22364', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e3506002-ba8b-4251-8ef1-136334574a6d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 22365}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 22365}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 22365}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 22365}]}, 'public_addr': '172.21.15.164:6817/22365', 'cluster_addr': '172.21.15.164:6819/22365', 'heartbeat_back_addr': '172.21.15.164:6823/22365', 'heartbeat_front_addr': '172.21.15.164:6821/22365', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.104631+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987403+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.018134+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987645+0000', 'dead_epoch': 59}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.721507+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.869796+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.206576+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.038098+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.7', 'mappings': [{'from': 0, 'to': 3}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T14:08:31.799 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T14:08:31.800 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:31.706+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:31.811 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:31.806+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:31.906 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:31.906 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:31.912 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:31.906+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:31.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:32.011 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:32.011 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:32.013 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:32.006+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:32.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:32.114 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:32.112+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:32.121 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.213629597999997, 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:08:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:08:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:08:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:08:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:08:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.213606890000001, 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:08:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.213582737999999, 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:08:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:08:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.213549796999999, 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:08:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.213525113999999, 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:08:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.213501884999999, 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:08:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.211655921000002, 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:08:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:08:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:08:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:08:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:08:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.211613057000001, 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:08:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:08:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210153439000003, 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:08:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:08:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:08:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210137670000002, 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210121917999999, 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:08:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210106486999997, 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:08:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:08:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210090848999997, 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:08:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:08:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210075709999998, 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:08:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210060873000003, 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:08:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210037868999997, 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:08:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.210022643999999, 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:08:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:08:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:08:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:08:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.21000746, 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:08:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.209992215, 2024-09-16T14:08:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.209977057000003, 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:08:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.233 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.234 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.234 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:32.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:32.236 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:32.232+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:32.336 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:32.330+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:32.395 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:32.395 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:32.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:32.410 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-16T14:08:32.426 INFO:teuthology.orchestra.run.smithi079.stdout:3757 write 0xc7e3709 thru 0xc7e429d (0xb95 bytes) 2024-09-16T14:08:32.436 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:32.430+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:32.443 INFO:teuthology.orchestra.run.smithi079.stdout:3758 read 0x2883067 thru 0x288e8d4 (0xb86e bytes) 2024-09-16T14:08:32.445 INFO:teuthology.orchestra.run.smithi079.stdout:3759 punch from 0x1c6953 to 0x1d54a9, (0xeb56 bytes) 2024-09-16T14:08:32.445 INFO:teuthology.orchestra.run.smithi079.stdout:3760 read 0x6844d8c thru 0x684ed48 (0x9fbd bytes) 2024-09-16T14:08:32.454 INFO:teuthology.orchestra.run.smithi079.stdout:3762 trunc from 0xde6a08c to 0xbcaa440 2024-09-16T14:08:32.506 INFO:teuthology.orchestra.run.smithi079.stdout:3764 trunc from 0xbcaa440 to 0x6581c4f 2024-09-16T14:08:32.523 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:32.523 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:32.523 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:32.523 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:32.523 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:32.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:32.537 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:32.532+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:32.563 INFO:teuthology.orchestra.run.smithi079.stdout:3765 write 0x8fc1ec4 thru 0x8fcd725 (0xb862 bytes) 2024-09-16T14:08:32.571 INFO:teuthology.orchestra.run.smithi079.stdout:3766 write 0x8152385 thru 0x8160c55 (0xe8d1 bytes) 2024-09-16T14:08:32.581 INFO:teuthology.orchestra.run.smithi079.stdout:3768 read 0x38d2bdf thru 0x38d342d (0x84f bytes) 2024-09-16T14:08:32.582 INFO:teuthology.orchestra.run.smithi079.stdout:3769 read 0x75e31e5 thru 0x75f0b7d (0xd999 bytes) 2024-09-16T14:08:32.582 INFO:teuthology.orchestra.run.smithi079.stdout:3770 read 0x56a9d86 thru 0x56b375b (0x99d6 bytes) 2024-09-16T14:08:32.600 INFO:teuthology.orchestra.run.smithi079.stdout:3771 writesame 0x93aeff5 thru 0x93b8a82 data_size 0x49(0x9a8e bytes) 2024-09-16T14:08:32.600 INFO:teuthology.orchestra.run.smithi079.stdout:3772 read 0x63b35c1 thru 0x63be213 (0xac53 bytes) 2024-09-16T14:08:32.601 INFO:teuthology.orchestra.run.smithi079.stdout:3773 write 0x5395009 thru 0x53a007d (0xb075 bytes) 2024-09-16T14:08:32.613 INFO:teuthology.orchestra.run.smithi079.stdout:3774 write 0x37cb47b thru 0x37cf67e (0x4204 bytes) 2024-09-16T14:08:32.621 INFO:teuthology.orchestra.run.smithi079.stdout:3775 trunc from 0x93b8a83 to 0x7e7b47b 2024-09-16T14:08:32.637 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:32.630+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:32.643 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:32.643 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:32.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:32.643 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:32.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86627 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 115712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.467866491000002, 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057990535000000003, 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86627 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.79e-07 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172073+0000", 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.53e-06 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172077+0000", 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172079+0000", 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224325+0000", 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052246133 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224394+0000", 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8960000000000004e-05 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.230062+0000", 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056682820000000002 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86989 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.446257645000003, 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058737710999999998, 2024-09-16T14:08:32.649 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86989 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193681+0000", 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193682+0000", 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6169999999999999e-06 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193687+0000", 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7210000000000002e-06 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193690+0000", 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.072e-06 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.246881+0000", 2024-09-16T14:08:32.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.053191117000000003 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252309+0000", 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054275979999999996 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252418+0000", 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109255 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86995 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.446231443000002, 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059640662999999997, 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.654 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86995 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197498+0000", 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037915219999999999 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197504+0000", 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7520000000000002e-06 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197515+0000", 2024-09-16T14:08:32.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.117e-05 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197521+0000", 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.083e-06 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253121+0000", 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055600254000000002 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253227+0000", 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010607000000000001 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253347+0000", 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119812 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87017 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.658 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442295707, 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676182999999998, 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87017 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197643+0000", 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197645+0000", 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197652+0000", 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8430000000000004e-06 2024-09-16T14:08:32.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197656+0000", 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254046+0000", 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056390033999999999 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254205+0000", 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158639 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254318+0000", 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113916 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87019 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442278131000002, 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056580459, 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87019 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197663+0000", 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.396e-06 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197669+0000", 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4099999999999996e-06 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197673+0000", 2024-09-16T14:08:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254066+0000", 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056392870999999997 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254160+0000", 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4574000000000006e-05 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254240+0000", 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0125999999999998e-05 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87023 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 193536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442261264000003, 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056736048999999997, 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87023 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197679+0000", 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197686+0000", 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197690+0000", 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.177e-06 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254263+0000", 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056573364000000001 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254334+0000", 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0574000000000006e-05 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254413+0000", 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079999999999995e-05 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87027 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442244141000003, 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056974745, 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87027 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2599999999999998e-07 2024-09-16T14:08:32.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197696+0000", 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1679999999999998e-06 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197703+0000", 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4389999999999997e-06 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197707+0000", 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254383+0000", 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676509999999999 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254483+0000", 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9692999999999996e-05 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254669+0000", 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018580300000000001 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87029 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 197632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442227303999999, 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057332283999999997, 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87029 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197713+0000", 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.199e-06 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197719+0000", 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.699e-06 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197723+0000", 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000002e-06 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254709+0000", 2024-09-16T14:08:32.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056986536999999997 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254790+0000", 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0192999999999996e-05 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255043+0000", 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025339500000000001 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87031 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 234496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442211012000001, 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057120535, 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87031 2024-09-16T14:08:32.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0600000000000001e-07 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197729+0000", 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0499999999999999e-06 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197736+0000", 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2249999999999997e-06 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197739+0000", 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6839999999999998e-06 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254697+0000", 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056957902999999997 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254776+0000", 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-16T14:08:32.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254848+0000", 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1044999999999995e-05 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87035 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 198656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442195007999999, 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057340290000000002, 2024-09-16T14:08:32.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87035 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197744+0000", 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6499999999999999e-07 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197746+0000", 2024-09-16T14:08:32.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0169999999999999e-06 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197752+0000", 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1040000000000003e-06 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197755+0000", 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.782e-06 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254869+0000", 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057113766000000003 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254971+0000", 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101932 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255083+0000", 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112124 2024-09-16T14:08:32.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87039 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 238592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442178597999998, 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057586228000000003, 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87039 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197760+0000", 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197762+0000", 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.829e-06 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197768+0000", 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5470000000000002e-06 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197772+0000", 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4699999999999998e-06 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255109+0000", 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057337232000000002 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255184+0000", 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5593000000000006e-05 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255346+0000", 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016120899999999999 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87041 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 202752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.695 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442162543999999, 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057827176000000001, 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87041 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197776+0000", 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:32.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197778+0000", 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3709999999999998e-06 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197784+0000", 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.054e-06 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197787+0000", 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1879999999999998e-06 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255397+0000", 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057609059999999997 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255483+0000", 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6415999999999994e-05 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255603+0000", 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119703 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87043 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 203776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442146377, 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057723202000000001, 2024-09-16T14:08:32.700 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87043 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197801+0000", 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5869999999999993e-06 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197807+0000", 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197811+0000", 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0099999999999997e-06 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255377+0000", 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057566063000000001 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255450+0000", 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3192e-05 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255515+0000", 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4543999999999997e-05 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87045 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442122183999999, 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057864405000000001, 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87045 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197819+0000", 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.311e-06 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197824+0000", 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6980000000000003e-06 2024-09-16T14:08:32.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197828+0000", 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2339999999999999e-06 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255537+0000", 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057709979000000002 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255613+0000", 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5807000000000001e-05 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255680+0000", 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6935999999999997e-05 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87051 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.442107032000003, 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058010645999999999, 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87051 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200527+0000", 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026964990000000002 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200532+0000", 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8899999999999998e-06 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200541+0000", 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1249999999999993e-06 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200546+0000", 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4129999999999997e-06 2024-09-16T14:08:32.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255630+0000", 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055083612999999997 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255711+0000", 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1934000000000005e-05 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255842+0000", 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013017200000000001 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87559 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 100352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.439272676000002, 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056509688000000002, 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87559 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200666+0000", 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200668+0000", 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.21e-06 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200674+0000", 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.896e-06 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200678+0000", 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7050000000000001e-06 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.256898+0000", 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056220883999999999 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257014+0000", 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011530499999999999 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257175+0000", 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016127200000000001 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87568 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.439240697000002, 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056739408999999998, 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87568 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200698+0000", 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200700+0000", 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.272e-06 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200706+0000", 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7080000000000002e-06 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200709+0000", 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.394e-06 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257221+0000", 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056512383999999999 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257319+0000", 2024-09-16T14:08:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7510000000000007e-05 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257437+0000", 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011773399999999999 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87570 3.e 3:73b2712a:::rbd_data.113619b3ee73.0000000000000027:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.439225364999999, 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056717201000000002, 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87570 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201471+0000", 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075828899999999999 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201480+0000", 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.448e-06 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201494+0000", 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3162999999999999e-05 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201500+0000", 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0090000000000004e-06 2024-09-16T14:08:32.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257205+0000", 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055705603999999999 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257330+0000", 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124972 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257430+0000", 2024-09-16T14:08:32.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9716000000000005e-05 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87773 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 63488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.433448351000003, 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056458201, 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87773 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207001+0000", 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051104000000000004 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207006+0000", 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207019+0000", 2024-09-16T14:08:32.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2266000000000001e-05 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207025+0000", 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2029999999999997e-06 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262711+0000", 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055686539 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262801+0000", 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9895999999999994e-05 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262948+0000", 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146642 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.1210402070000001, 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:32.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:08:32.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:08:32.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:32.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:32.738 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:32.739 INFO:teuthology.orchestra.run.smithi079.stdout:3779 trunc from 0x7e7b47b to 0xe5a6e83 2024-09-16T14:08:32.740 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3780 read 0x5016f51 thru 0x50179c0 (0xa70 bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3783 punch from 0x695200d to 0x6961669, (0xf65c bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3784 trunc from 0xe5a6e83 to 0x9eba693 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3785 write 0x3905d4b thru 0x390788d (0x1b43 bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3786 read 0x6d3a0cf thru 0x6d4804e (0xdf80 bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3788 write 0x1da7694 thru 0x1daf558 (0x7ec5 bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3789 punch from 0x16bafa7 to 0x16bd316, (0x236f bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3790 trunc from 0x9eba693 to 0x7fe38fb 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3791 punch from 0x4fc7c61 to 0x4fcd7c1, (0x5b60 bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3792 read 0x7a6c38a thru 0x7a78c15 (0xc88c bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3793 punch from 0x708756 to 0x70b2e0, (0x2b8a bytes) 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:32.742 INFO:teuthology.orchestra.run.smithi079.stdout:3795 write 0x5718ef2 thru 0x5726496 (0xd5a5 bytes) 2024-09-16T14:08:32.743 INFO:teuthology.orchestra.run.smithi079.stdout:3796 read 0x5ae8b1d thru 0x5aeb95a (0x2e3e bytes) 2024-09-16T14:08:32.743 INFO:teuthology.orchestra.run.smithi079.stdout:3797 read 0x671741e thru 0x671d329 (0x5f0c bytes) 2024-09-16T14:08:32.743 INFO:teuthology.orchestra.run.smithi079.stdout:3798 write 0x2d83729 thru 0x2d86ea7 (0x377f bytes) 2024-09-16T14:08:32.743 INFO:teuthology.orchestra.run.smithi079.stdout:3799 read 0x7d135ab thru 0x7d22142 (0xeb98 bytes) 2024-09-16T14:08:32.743 INFO:teuthology.orchestra.run.smithi079.stdout:3800 punch from 0x4437345 to 0x4446405, (0xf0c0 bytes) 2024-09-16T14:08:32.743 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:32.743 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:32.736+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:32.743 INFO:teuthology.orchestra.run.smithi079.stdout:3802 trunc from 0x7fe38fb to 0x62c66cf 2024-09-16T14:08:32.756 INFO:teuthology.orchestra.run.smithi079.stdout:3803 punch from 0x22eed5c to 0x22fbc40, (0xcee4 bytes) 2024-09-16T14:08:32.756 INFO:teuthology.orchestra.run.smithi079.stdout:3804 write 0x58ae1e3 thru 0x58bc7b5 (0xe5d3 bytes) 2024-09-16T14:08:32.789 INFO:teuthology.orchestra.run.smithi079.stdout:3805 writesame 0xb82a5a5 thru 0xb837a45 data_size 0x1(0xd4a1 bytes) 2024-09-16T14:08:32.789 INFO:teuthology.orchestra.run.smithi079.stdout:3806 punch from 0x1e1df75 to 0x1e29312, (0xb39d bytes) 2024-09-16T14:08:32.803 INFO:teuthology.orchestra.run.smithi079.stdout:3808 write 0xc51ba09 thru 0xc51ca97 (0x108f bytes) 2024-09-16T14:08:32.810 INFO:teuthology.orchestra.run.smithi079.stdout:3809 punch from 0x82d92b2 to 0x82d9eb9, (0xc07 bytes) 2024-09-16T14:08:32.810 INFO:teuthology.orchestra.run.smithi079.stdout:3810 punch from 0x72b461a to 0x72ba90f, (0x62f5 bytes) 2024-09-16T14:08:32.810 INFO:teuthology.orchestra.run.smithi079.stdout:3812 read 0x495b09a thru 0x495c1b3 (0x111a bytes) 2024-09-16T14:08:32.812 INFO:teuthology.orchestra.run.smithi079.stdout:3814 write 0x1f8b88f thru 0x1f9a39a (0xeb0c bytes) 2024-09-16T14:08:32.830 INFO:teuthology.orchestra.run.smithi079.stdout:3815 read 0x21c78d9 thru 0x21d4e14 (0xd53c bytes) 2024-09-16T14:08:32.833 INFO:teuthology.orchestra.run.smithi079.stdout:3816 read 0x4914e6 thru 0x49bd85 (0xa8a0 bytes) 2024-09-16T14:08:32.841 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:32.836+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:32.845 INFO:teuthology.orchestra.run.smithi079.stdout:3817 trunc from 0xc51ca98 to 0xa0db76b 2024-09-16T14:08:32.854 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:32.854 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:32.864 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:32.869 INFO:teuthology.orchestra.run.smithi079.stdout:3819 punch from 0x3f3cce7 to 0x3f3d729, (0xa42 bytes) 2024-09-16T14:08:32.869 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:32.869 INFO:teuthology.orchestra.run.smithi079.stdout:3821 write 0x5f72ec1 thru 0x5f798ed (0x6a2d bytes) 2024-09-16T14:08:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:3822 punch from 0x5f6a579 to 0x5f6cb6d, (0x25f4 bytes) 2024-09-16T14:08:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:3824 read 0x1ccecd0 thru 0x1cdb48c (0xc7bd bytes) 2024-09-16T14:08:32.875 INFO:teuthology.orchestra.run.smithi079.stdout:3825 read 0x4cf8266 thru 0x4cf9166 (0xf01 bytes) 2024-09-16T14:08:32.877 INFO:teuthology.orchestra.run.smithi079.stdout:3827 writesame 0x257ae70 thru 0x257ca6a data_size 0x13(0x1bfb bytes) 2024-09-16T14:08:32.887 INFO:teuthology.orchestra.run.smithi079.stdout:3829 write 0x8145b33 thru 0x814fe56 (0xa324 bytes) 2024-09-16T14:08:32.893 INFO:teuthology.orchestra.run.smithi079.stdout:3830 read 0x214bdac thru 0x2159212 (0xd467 bytes) 2024-09-16T14:08:32.894 INFO:teuthology.orchestra.run.smithi079.stdout:3832 write 0x75a43a2 thru 0x75aa2f9 (0x5f58 bytes) 2024-09-16T14:08:32.902 INFO:teuthology.orchestra.run.smithi079.stdout:3837 punch from 0x99690d6 to 0x996aa4b, (0x1975 bytes) 2024-09-16T14:08:32.902 INFO:teuthology.orchestra.run.smithi079.stdout:3838 read 0x6330146 thru 0x633344e (0x3309 bytes) 2024-09-16T14:08:32.902 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:32.908 INFO:teuthology.orchestra.run.smithi079.stdout:3840 trunc from 0xa0db76b to 0xd2da85f 2024-09-16T14:08:32.919 INFO:teuthology.orchestra.run.smithi079.stdout:3841 punch from 0x101809c to 0x101ab59, (0x2abd bytes) 2024-09-16T14:08:32.919 INFO:teuthology.orchestra.run.smithi079.stdout:3842 write 0x78dd5ab thru 0x78df6f9 (0x214f bytes) 2024-09-16T14:08:32.925 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:32.941 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:32.936+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:32.947 INFO:teuthology.orchestra.run.smithi079.stdout:3844 trunc from 0xd2da85f to 0x26fccea 2024-09-16T14:08:32.960 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:32.960 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:32.960 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:32.960 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:32.960 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:32.971 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:32.989 INFO:teuthology.orchestra.run.smithi079.stdout:3845 trunc from 0x26fccea to 0x490ecec 2024-09-16T14:08:32.998 INFO:teuthology.orchestra.run.smithi079.stdout:3847 read 0x2caeee5 thru 0x2cafc22 (0xd3e bytes) 2024-09-16T14:08:32.998 INFO:teuthology.orchestra.run.smithi079.stdout:3852 write 0x1cd1e68 thru 0x1cda340 (0x84d9 bytes) 2024-09-16T14:08:33.001 INFO:teuthology.orchestra.run.smithi079.stdout:3853 read 0x11c0d6f thru 0x11c8e9a (0x812c bytes) 2024-09-16T14:08:33.023 INFO:teuthology.orchestra.run.smithi079.stdout:3855 write 0x87254f0 thru 0x872c1a2 (0x6cb3 bytes) 2024-09-16T14:08:33.032 INFO:teuthology.orchestra.run.smithi079.stdout:3856 trunc from 0x872c1a3 to 0x63da8d4 2024-09-16T14:08:33.042 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:33.034+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:33.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:33.072 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:33.072 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:33.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.731571711000001, 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.073 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:33.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:33.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.731549314000006, 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:33.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:33.085 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.730942603000003, 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.086 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:33.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:33.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.730868275999995, 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.093 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:33.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.098 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.759943484000004, 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:33.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:33.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:33.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:33.103 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.131984903000003, 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:33.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:33.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:33.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.601824436999998, 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:33.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:33.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:33.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:33.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.954275905999999, 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.117 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101781 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.408030112999999, 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019058275, 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101781 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660443+0000", 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660450+0000", 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999997e-06 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660454+0000", 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0029999999999996e-06 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679188+0000", 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018734869000000001 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679271+0000", 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2721999999999998e-05 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679499+0000", 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227612 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101783 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.408012934999999, 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019291800000000001, 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101783 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4499999999999997e-07 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660465+0000", 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4659999999999996e-06 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660471+0000", 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5250000000000002e-06 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660480+0000", 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5779999999999999e-06 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679530+0000", 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019050085000000001 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679633+0000", 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103341 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679750+0000", 2024-09-16T14:08:33.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011636 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.407985665999998, 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.131 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:33.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:33.135 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.407968475000001, 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:33.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:33.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.407931458, 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.140 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:33.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.144 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.407910781999998, 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:33.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:33.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.407868602000001, 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.153 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.407834377, 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.154 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:33.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:33.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.407804661, 2024-09-16T14:08:33.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:33.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.407784999, 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:33.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:33.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.399024873999998, 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:33.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:33.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.314298332, 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.173 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:08:33.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:33.177 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:33.179 INFO:teuthology.orchestra.run.smithi079.stdout:3857 write 0xdb239ee thru 0xdb30ee3 (0xd4f6 bytes) 2024-09-16T14:08:33.179 INFO:teuthology.orchestra.run.smithi079.stdout:3858 write 0xe5734a8 thru 0xe580caa (0xd803 bytes) 2024-09-16T14:08:33.179 INFO:teuthology.orchestra.run.smithi079.stdout:3860 trunc from 0xe580cab to 0x73b507c 2024-09-16T14:08:33.179 INFO:teuthology.orchestra.run.smithi079.stdout:3861 write 0x551e2f0 thru 0x5523912 (0x5623 bytes) 2024-09-16T14:08:33.180 INFO:teuthology.orchestra.run.smithi079.stdout:3864 read 0x6df1f6d thru 0x6dfdfa9 (0xc03d bytes) 2024-09-16T14:08:33.180 INFO:teuthology.orchestra.run.smithi079.stdout:3865 write 0x3ff2e97 thru 0x3ff404f (0x11b9 bytes) 2024-09-16T14:08:33.180 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:33.180 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:33.176+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:33.181 INFO:teuthology.orchestra.run.smithi079.stdout:3866 write 0xb8412ad thru 0xb84e266 (0xcfba bytes) 2024-09-16T14:08:33.188 INFO:teuthology.orchestra.run.smithi079.stdout:3867 writesame 0x7b40666 thru 0x7b429e1 data_size 0x6(0x237c bytes) 2024-09-16T14:08:33.195 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:33.213 INFO:teuthology.orchestra.run.smithi079.stdout:3868 writesame 0xddde229 thru 0xdde830d data_size 0x1(0xa0e5 bytes) 2024-09-16T14:08:33.215 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:33.220 INFO:teuthology.orchestra.run.smithi079.stdout:3870 trunc from 0xdde830e to 0x9ca5fb8 2024-09-16T14:08:33.234 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:33.246 INFO:teuthology.orchestra.run.smithi079.stdout:3871 trunc from 0x9ca5fb8 to 0x8967429 2024-09-16T14:08:33.251 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:33.267 INFO:teuthology.orchestra.run.smithi079.stdout:3872 trunc from 0x8967429 to 0x4214bdd 2024-09-16T14:08:33.279 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:33.274+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:33.283 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:33.289 INFO:teuthology.orchestra.run.smithi079.stdout:3873 writesame 0xf82f43 thru 0xf8bff2 data_size 0x28(0x90b0 bytes) 2024-09-16T14:08:33.303 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:33.303 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:33.309 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:33.314 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:33.315 INFO:teuthology.orchestra.run.smithi079.stdout:3874 write 0x447fc4c thru 0x4481a53 (0x1e08 bytes) 2024-09-16T14:08:33.323 INFO:teuthology.orchestra.run.smithi079.stdout:3876 read 0x1f69727 thru 0x1f6c1ac (0x2a86 bytes) 2024-09-16T14:08:33.324 INFO:teuthology.orchestra.run.smithi079.stdout:3877 read 0x2c0edea thru 0x2c1c870 (0xda87 bytes) 2024-09-16T14:08:33.332 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:33.355 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:33.356 INFO:teuthology.orchestra.run.smithi079.stdout:3879 writesame 0xe980985 thru 0xe9877f8 data_size 0x2(0x6e74 bytes) 2024-09-16T14:08:33.358 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:33.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:33.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:33.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:33.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:33.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:33.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:33.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:33.362 INFO:teuthology.orchestra.run.smithi079.stdout:3880 trunc from 0xe9877f9 to 0xba0e46a 2024-09-16T14:08:33.373 INFO:teuthology.orchestra.run.smithi079.stdout:3881 writesame 0xb55bb0e thru 0xb55cb82 data_size 0x1(0x1075 bytes) 2024-09-16T14:08:33.378 INFO:teuthology.orchestra.run.smithi079.stdout:3882 punch from 0x6fbd2cf to 0x6fc1d05, (0x4a36 bytes) 2024-09-16T14:08:33.378 INFO:teuthology.orchestra.run.smithi079.stdout:3883 writesame 0xcd2f1b thru 0xcdda4d data_size 0x7(0xab33 bytes) 2024-09-16T14:08:33.380 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:33.374+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:33.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:08:33.397 INFO:teuthology.orchestra.run.smithi079.stdout:3886 write 0x9bc0666 thru 0x9bcfe0c (0xf7a7 bytes) 2024-09-16T14:08:33.405 INFO:teuthology.orchestra.run.smithi079.stdout:3887 read 0x14f88ef thru 0x14fcf1e (0x4630 bytes) 2024-09-16T14:08:33.406 INFO:teuthology.orchestra.run.smithi079.stdout:3889 write 0x715aaa1 thru 0x71684e1 (0xda41 bytes) 2024-09-16T14:08:33.420 INFO:teuthology.orchestra.run.smithi079.stdout:3890 read 0x41321c8 thru 0x413c40d (0xa246 bytes) 2024-09-16T14:08:33.420 INFO:teuthology.orchestra.run.smithi079.stdout:3891 write 0x4b76ba3 thru 0x4b7f3e6 (0x8844 bytes) 2024-09-16T14:08:33.426 INFO:teuthology.orchestra.run.smithi079.stdout:3892 read 0x1f0d8f3 thru 0x1f1318a (0x5898 bytes) 2024-09-16T14:08:33.462 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:33.462 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:33.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:33.480 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:33.474+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:33.566 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:33.566 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:33.575 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:33.581 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:33.576+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:33.612 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:33.612 INFO:teuthology.orchestra.run.smithi079.stdout:3894 read 0x141e52 thru 0x14e72b (0xc8da bytes) 2024-09-16T14:08:33.615 INFO:teuthology.orchestra.run.smithi079.stdout:3896 read 0x9eaa6ea thru 0x9eb6184 (0xba9b bytes) 2024-09-16T14:08:33.624 INFO:teuthology.orchestra.run.smithi079.stdout:3897 trunc from 0xba0e46a to 0x92f388c 2024-09-16T14:08:33.649 INFO:teuthology.orchestra.run.smithi079.stdout:3898 writesame 0x6bc4835 thru 0x6bd3a39 data_size 0x35(0xf205 bytes) 2024-09-16T14:08:33.659 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:33.659 INFO:teuthology.orchestra.run.smithi079.stdout:3900 writesame 0x16e1465 thru 0x16efd37 data_size 0x1(0xe8d3 bytes) 2024-09-16T14:08:33.682 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:33.674+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:33.690 INFO:teuthology.orchestra.run.smithi079.stdout:3901 read 0x78c775d thru 0x78ce0ff (0x69a3 bytes) 2024-09-16T14:08:33.690 INFO:teuthology.orchestra.run.smithi079.stdout:3902 writesame 0x7b5d1d4 thru 0x7b6a48c data_size 0x1(0xd2b9 bytes) 2024-09-16T14:08:33.719 INFO:teuthology.orchestra.run.smithi079.stdout:3903 read 0x35195f2 thru 0x3522aa4 (0x94b3 bytes) 2024-09-16T14:08:33.719 INFO:teuthology.orchestra.run.smithi079.stdout:3904 writesame 0x7787720 thru 0x7789446 data_size 0x1(0x1d27 bytes) 2024-09-16T14:08:33.720 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:33.720 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:33.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:33.749 INFO:teuthology.orchestra.run.smithi079.stdout:3906 write 0xcc3d495 thru 0xcc46217 (0x8d83 bytes) 2024-09-16T14:08:33.758 INFO:teuthology.orchestra.run.smithi079.stdout:3909 writesame 0x289f822 thru 0x28a30d5 data_size 0x26(0x38b4 bytes) 2024-09-16T14:08:33.766 INFO:teuthology.orchestra.run.smithi079.stdout:3910 read 0xade8a50 thru 0xadf772d (0xecde bytes) 2024-09-16T14:08:33.766 INFO:teuthology.orchestra.run.smithi079.stdout:3912 write 0xa94b1e3 thru 0xa94ea48 (0x3866 bytes) 2024-09-16T14:08:33.771 INFO:teuthology.orchestra.run.smithi079.stdout:3913 read 0x3035e86 thru 0x303f112 (0x928d bytes) 2024-09-16T14:08:33.771 INFO:teuthology.orchestra.run.smithi079.stdout:3914 writesame 0x27d8e2 thru 0x28a045 data_size 0xe(0xc764 bytes) 2024-09-16T14:08:33.782 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:33.782 INFO:teuthology.orchestra.run.smithi079.stdout:3916 write 0xac9d963 thru 0xaca497a (0x7018 bytes) 2024-09-16T14:08:33.783 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:33.778+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:33.789 INFO:teuthology.orchestra.run.smithi079.stdout:3919 punch from 0x6e792f0 to 0x6e84f3c, (0xbc4c bytes) 2024-09-16T14:08:33.789 INFO:teuthology.orchestra.run.smithi079.stdout:3920 writesame 0x3d08db9 thru 0x3d0d632 data_size 0x1(0x487a bytes) 2024-09-16T14:08:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:3923 clone 34 order 24 su 1048576 sc 2 2024-09-16T14:08:33.823 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:33.823 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:33.832 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:33.884 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:33.880+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:33.936 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:33.984 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:33.978+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:34.035 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.035 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.044 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:34.085 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:34.080+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:34.139 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.139 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.148 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:34.186 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:34.184+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:34.242 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.242 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:34.287 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:34.282+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:34.361 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.361 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:34.375 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:08:34.387 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:34.384+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:34.463 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.463 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:34.488 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:34.484+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:34.560 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone33 from 0xcc46218 (overlap 0x26fccea) to 0x137e675 2024-09-16T14:08:34.588 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:34.584+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:34.590 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.590 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:34.688 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:34.684+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:34.752 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.752 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:34.810 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:34.808+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:34.842 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-16T14:08:34.864 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.864 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:34.889 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:34.884+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:34.938 INFO:teuthology.orchestra.run.smithi079.stdout:3924 read 0xc8d58e0 thru 0xc8d91b4 (0x38d5 bytes) 2024-09-16T14:08:34.948 INFO:teuthology.orchestra.run.smithi079.stdout:3925 writesame 0x7a7ab3 thru 0x7ad439 data_size 0x29(0x5987 bytes) 2024-09-16T14:08:34.984 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:34.984 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:34.991 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:34.986+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:34.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:35.025 INFO:teuthology.orchestra.run.smithi079.stdout:3926 write 0x4e98b5b thru 0x4ea1372 (0x8818 bytes) 2024-09-16T14:08:35.036 INFO:teuthology.orchestra.run.smithi079.stdout:3928 write 0x6a5f9cf thru 0x6a6a382 (0xa9b4 bytes) 2024-09-16T14:08:35.056 INFO:teuthology.orchestra.run.smithi079.stdout:3929 trunc from 0xcc46218 to 0x6a3a51d 2024-09-16T14:08:35.088 INFO:teuthology.orchestra.run.smithi079.stdout:3930 punch from 0x3ee0979 to 0x3ee41c2, (0x3849 bytes) 2024-09-16T14:08:35.088 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:35.088 INFO:teuthology.orchestra.run.smithi079.stdout:3932 trunc from 0x6a3a51d to 0x6946c1d 2024-09-16T14:08:35.091 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:35.086+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:35.100 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:35.101 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:35.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:35.118 INFO:teuthology.orchestra.run.smithi079.stdout:3933 writesame 0xa8d80ad thru 0xa8e21c8 data_size 0x2a(0xa11c bytes) 2024-09-16T14:08:35.118 INFO:teuthology.orchestra.run.smithi079.stdout:3934 write 0x8e05f1 thru 0x8ec6b6 (0xc0c6 bytes) 2024-09-16T14:08:35.193 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:35.188+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:35.193 INFO:teuthology.orchestra.run.smithi079.stdout:3935 write 0x564a2d1 thru 0x5659173 (0xeea3 bytes) 2024-09-16T14:08:35.211 INFO:teuthology.orchestra.run.smithi079.stdout:3937 writesame 0xbdb5bd5 thru 0xbdbe16e data_size 0x31(0x859a bytes) 2024-09-16T14:08:35.211 INFO:teuthology.orchestra.run.smithi079.stdout:3938 read 0x33e60dc thru 0x33e78e4 (0x1809 bytes) 2024-09-16T14:08:35.214 INFO:teuthology.orchestra.run.smithi079.stdout:3939 writesame 0xad0660f thru 0xad0b531 data_size 0x3(0x4f23 bytes) 2024-09-16T14:08:35.218 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:35.219 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:35.224 INFO:teuthology.orchestra.run.smithi079.stdout:3941 writesame 0x3dab391 thru 0x3db79e3 data_size 0x1(0xc653 bytes) 2024-09-16T14:08:35.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:35.272 INFO:teuthology.orchestra.run.smithi079.stdout:3943 writesame 0x4553371 thru 0x4554852 data_size 0x21(0x14e2 bytes) 2024-09-16T14:08:35.286 INFO:teuthology.orchestra.run.smithi079.stdout:3944 read 0x501c843 thru 0x502ae56 (0xe614 bytes) 2024-09-16T14:08:35.288 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:35.288 INFO:teuthology.orchestra.run.smithi079.stdout:3950 punch from 0x5f0b7c8 to 0x5f17255, (0xba8d bytes) 2024-09-16T14:08:35.288 INFO:teuthology.orchestra.run.smithi079.stdout:3952 read 0xaeacc78 thru 0xaeb1f66 (0x52ef bytes) 2024-09-16T14:08:35.293 INFO:teuthology.orchestra.run.smithi079.stdout:3953 trunc from 0xbdbe16f to 0x8ee90d2 2024-09-16T14:08:35.293 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:35.286+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:35.315 INFO:teuthology.orchestra.run.smithi079.stdout:3956 punch from 0x883a4e5 to 0x88478f7, (0xd412 bytes) 2024-09-16T14:08:35.315 INFO:teuthology.orchestra.run.smithi079.stdout:3957 read 0xe5a7d6 thru 0xe65518 (0xad43 bytes) 2024-09-16T14:08:35.316 INFO:teuthology.orchestra.run.smithi079.stdout:3958 write 0x78dcc76 thru 0x78e7526 (0xa8b1 bytes) 2024-09-16T14:08:35.320 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:35.320 INFO:teuthology.orchestra.run.smithi079.stdout:3961 write 0xc7f2e5 thru 0xc8b0b6 (0xbdd2 bytes) 2024-09-16T14:08:35.325 INFO:teuthology.orchestra.run.smithi079.stdout:3963 read 0x5c60f62 thru 0x5c64fad (0x404c bytes) 2024-09-16T14:08:35.326 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:35.326 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:35.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:35.346 INFO:teuthology.orchestra.run.smithi079.stdout:3964 write 0xd941bfd thru 0xd941ebd (0x2c1 bytes) 2024-09-16T14:08:35.353 INFO:teuthology.orchestra.run.smithi079.stdout:3966 write 0x77e332d thru 0x77f1da4 (0xea78 bytes) 2024-09-16T14:08:35.357 INFO:teuthology.orchestra.run.smithi079.stdout:3967 punch from 0x21f06e1 to 0x21fd5ab, (0xceca bytes) 2024-09-16T14:08:35.357 INFO:teuthology.orchestra.run.smithi079.stdout:3970 read 0xb77e917 thru 0xb780f40 (0x262a bytes) 2024-09-16T14:08:35.357 INFO:teuthology.orchestra.run.smithi079.stdout:3971 punch from 0xcd6bab7 to 0xcd7897c, (0xcec5 bytes) 2024-09-16T14:08:35.357 INFO:teuthology.orchestra.run.smithi079.stdout:3976 punch from 0x65f8a7d to 0x6607af9, (0xf07c bytes) 2024-09-16T14:08:35.357 INFO:teuthology.orchestra.run.smithi079.stdout:3977 read 0x56a7d91 thru 0x56a8d3c (0xfac bytes) 2024-09-16T14:08:35.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:35.378 INFO:teuthology.orchestra.run.smithi079.stdout:3978 trunc from 0xd941ebe to 0xd86949 2024-09-16T14:08:35.394 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:35.392+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:35.423 INFO:teuthology.orchestra.run.smithi079.stdout:3981 write 0xeb988c thru 0xec2212 (0x8987 bytes) 2024-09-16T14:08:35.442 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:35.442 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:35.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:35.455 INFO:teuthology.orchestra.run.smithi079.stdout:3982 writesame 0xd9d1a17 thru 0xd9df5fd data_size 0x87(0xdbe7 bytes) 2024-09-16T14:08:35.455 INFO:teuthology.orchestra.run.smithi079.stdout:3983 writesame 0xbefbe4e thru 0xbf00032 data_size 0x1(0x41e5 bytes) 2024-09-16T14:08:35.478 INFO:teuthology.orchestra.run.smithi079.stdout:3984 punch from 0xb58cc72 to 0xb59a199, (0xd527 bytes) 2024-09-16T14:08:35.478 INFO:teuthology.orchestra.run.smithi079.stdout:3985 write 0xc18544c thru 0xc18758c (0x2141 bytes) 2024-09-16T14:08:35.483 INFO:teuthology.orchestra.run.smithi079.stdout:3989 read 0x8c6ecc thru 0x8d1969 (0xaa9e bytes) 2024-09-16T14:08:35.485 INFO:teuthology.orchestra.run.smithi079.stdout:3991 read 0x70d5c0c thru 0x70d857f (0x2974 bytes) 2024-09-16T14:08:35.485 INFO:teuthology.orchestra.run.smithi079.stdout:3996 read 0x5c6b5e7 thru 0x5c734b1 (0x7ecb bytes) 2024-09-16T14:08:35.485 INFO:teuthology.orchestra.run.smithi079.stdout:3997 writesame 0x42619cb thru 0x426c883 data_size 0x1(0xaeb9 bytes) 2024-09-16T14:08:35.494 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:35.492+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:35.504 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:35.517 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:35.532 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:35.540 INFO:teuthology.orchestra.run.smithi079.stdout:3999 trunc from 0xd9df5fe to 0xd5acd9 2024-09-16T14:08:35.551 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:35.558 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:35.558 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:35.558 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:35.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:35.569 INFO:teuthology.orchestra.run.smithi079.stdout:4000 trunc from 0xd5acd9 to 0x4710631 2024-09-16T14:08:35.575 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:35.593 INFO:teuthology.orchestra.run.smithi079.stdout:4001 trunc from 0x4710631 to 0xcc0aaa3 2024-09-16T14:08:35.596 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:35.590+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:35.604 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:35.605 INFO:teuthology.orchestra.run.smithi079.stdout:4002 writesame 0x8085be2 thru 0x8090a01 data_size 0x38(0xae20 bytes) 2024-09-16T14:08:35.613 INFO:teuthology.orchestra.run.smithi079.stdout:4004 clone 35 order 24 su 131072 sc 7 2024-09-16T14:08:35.630 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:35.650 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:35.666 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86627 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 115712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:35.666 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.489970790000001, 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057990535000000003, 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86627 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.79e-07 2024-09-16T14:08:35.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172073+0000", 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.53e-06 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172077+0000", 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-16T14:08:35.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172079+0000", 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224325+0000", 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052246133 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224394+0000", 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8960000000000004e-05 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.230062+0000", 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056682820000000002 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86989 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.671 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.468361944000002, 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058737710999999998, 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86989 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193681+0000", 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:35.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193682+0000", 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6169999999999999e-06 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193687+0000", 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7210000000000002e-06 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193690+0000", 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.072e-06 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.246881+0000", 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.053191117000000003 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252309+0000", 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054275979999999996 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252418+0000", 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109255 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86995 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.468335742000001, 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059640662999999997, 2024-09-16T14:08:35.676 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86995 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197498+0000", 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037915219999999999 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197504+0000", 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7520000000000002e-06 2024-09-16T14:08:35.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197515+0000", 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.117e-05 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197521+0000", 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.083e-06 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253121+0000", 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055600254000000002 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253227+0000", 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010607000000000001 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253347+0000", 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119812 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.680 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87017 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464400005999998, 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676182999999998, 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.681 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87017 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197643+0000", 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197645+0000", 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197652+0000", 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8430000000000004e-06 2024-09-16T14:08:35.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197656+0000", 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254046+0000", 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056390033999999999 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254205+0000", 2024-09-16T14:08:35.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158639 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254318+0000", 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113916 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87019 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:35.685 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464382430000001, 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056580459, 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87019 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:35.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197663+0000", 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.396e-06 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197669+0000", 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4099999999999996e-06 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197673+0000", 2024-09-16T14:08:35.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254066+0000", 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056392870999999997 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254160+0000", 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4574000000000006e-05 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254240+0000", 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0125999999999998e-05 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87023 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 193536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464365563000001, 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056736048999999997, 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87023 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197679+0000", 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197686+0000", 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197690+0000", 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.177e-06 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254263+0000", 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056573364000000001 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254334+0000", 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0574000000000006e-05 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254413+0000", 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079999999999995e-05 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87027 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464348440000002, 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056974745, 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87027 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2599999999999998e-07 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197696+0000", 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1679999999999998e-06 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197703+0000", 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4389999999999997e-06 2024-09-16T14:08:35.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197707+0000", 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254383+0000", 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676509999999999 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254483+0000", 2024-09-16T14:08:35.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9692999999999996e-05 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254669+0000", 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018580300000000001 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87029 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 197632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:35.699 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464331602999998, 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057332283999999997, 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87029 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197713+0000", 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.199e-06 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197719+0000", 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.699e-06 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197723+0000", 2024-09-16T14:08:35.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000002e-06 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254709+0000", 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056986536999999997 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254790+0000", 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0192999999999996e-05 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255043+0000", 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025339500000000001 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87031 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 234496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464315311, 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057120535, 2024-09-16T14:08:35.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87031 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0600000000000001e-07 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197729+0000", 2024-09-16T14:08:35.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0499999999999999e-06 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197736+0000", 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2249999999999997e-06 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197739+0000", 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6839999999999998e-06 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254697+0000", 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056957902999999997 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254776+0000", 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254848+0000", 2024-09-16T14:08:35.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1044999999999995e-05 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87035 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 198656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464299306999997, 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057340290000000002, 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87035 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197744+0000", 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6499999999999999e-07 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197746+0000", 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0169999999999999e-06 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197752+0000", 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1040000000000003e-06 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197755+0000", 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.782e-06 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254869+0000", 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057113766000000003 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254971+0000", 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101932 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255083+0000", 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112124 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87039 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 238592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464282896999997, 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057586228000000003, 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87039 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197760+0000", 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197762+0000", 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.829e-06 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197768+0000", 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5470000000000002e-06 2024-09-16T14:08:35.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197772+0000", 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4699999999999998e-06 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255109+0000", 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057337232000000002 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255184+0000", 2024-09-16T14:08:35.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5593000000000006e-05 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255346+0000", 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016120899999999999 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87041 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 202752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:35.718 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464266842999997, 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057827176000000001, 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87041 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197776+0000", 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197778+0000", 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3709999999999998e-06 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197784+0000", 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.054e-06 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197787+0000", 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1879999999999998e-06 2024-09-16T14:08:35.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255397+0000", 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057609059999999997 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255483+0000", 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6415999999999994e-05 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255603+0000", 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119703 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87043 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 203776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464250675999999, 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057723202000000001, 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87043 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197801+0000", 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5869999999999993e-06 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197807+0000", 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197811+0000", 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0099999999999997e-06 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255377+0000", 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057566063000000001 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255450+0000", 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3192e-05 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255515+0000", 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4543999999999997e-05 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.727 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87045 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464226482999997, 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057864405000000001, 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87045 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197819+0000", 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.311e-06 2024-09-16T14:08:35.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197824+0000", 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6980000000000003e-06 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197828+0000", 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2339999999999999e-06 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255537+0000", 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057709979000000002 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255613+0000", 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5807000000000001e-05 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255680+0000", 2024-09-16T14:08:35.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6935999999999997e-05 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87051 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.464211331000001, 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058010645999999999, 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87051 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:35.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200527+0000", 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026964990000000002 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200532+0000", 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8899999999999998e-06 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200541+0000", 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1249999999999993e-06 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200546+0000", 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4129999999999997e-06 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255630+0000", 2024-09-16T14:08:35.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055083612999999997 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255711+0000", 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1934000000000005e-05 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255842+0000", 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013017200000000001 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87559 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 100352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.461376975, 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056509688000000002, 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87559 2024-09-16T14:08:35.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200666+0000", 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200668+0000", 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.21e-06 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200674+0000", 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.896e-06 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200678+0000", 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7050000000000001e-06 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.256898+0000", 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056220883999999999 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257014+0000", 2024-09-16T14:08:35.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011530499999999999 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257175+0000", 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016127200000000001 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87568 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:35.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.461344996000001, 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056739408999999998, 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87568 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:35.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200698+0000", 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200700+0000", 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.272e-06 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200706+0000", 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7080000000000002e-06 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200709+0000", 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.394e-06 2024-09-16T14:08:35.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257221+0000", 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056512383999999999 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257319+0000", 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7510000000000007e-05 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257437+0000", 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011773399999999999 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87570 3.e 3:73b2712a:::rbd_data.113619b3ee73.0000000000000027:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.461329663999997, 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056717201000000002, 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87570 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:35.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201471+0000", 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075828899999999999 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201480+0000", 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.448e-06 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201494+0000", 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3162999999999999e-05 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201500+0000", 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0090000000000004e-06 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257205+0000", 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055705603999999999 2024-09-16T14:08:35.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257330+0000", 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124972 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257430+0000", 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9716000000000005e-05 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87773 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 63488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.455552650000001, 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056458201, 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87773 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.206490+0000", 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207001+0000", 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00051104000000000004 2024-09-16T14:08:35.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207006+0000", 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6149999999999996e-06 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207019+0000", 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2266000000000001e-05 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.207025+0000", 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2029999999999997e-06 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262711+0000", 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055686539 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262801+0000", 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9895999999999994e-05 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.262948+0000", 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000146642 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.1431445059999996, 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:08:35.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:08:35.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:35.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:35.762 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:35.762 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:35.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:35.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:35.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:35.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:35.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:35.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:35.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:35.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:35.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:35.765 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:35.760+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:35.864 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:35.860+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:35.874 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:35.874 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:35.875 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:35.875 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:35.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:35.965 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:35.958+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:35.982 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:35.982 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:35.982 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:35.982 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:35.982 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:35.991 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:36.066 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:36.058+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.754433582999994, 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:36.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.754411185999999, 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.102 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:36.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:36.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.753804474999995, 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.108 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:36.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:36.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.753730148000002, 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.114 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:36.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:36.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:36.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.782805355999997, 2024-09-16T14:08:36.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:36.124 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.154846774999996, 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.125 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:36.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.624686308999998, 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:36.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:36.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.977137777999999, 2024-09-16T14:08:36.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:36.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:36.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101781 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:36.142 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430891985000002, 2024-09-16T14:08:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019058275, 2024-09-16T14:08:36.143 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101781 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660441+0000", 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2899999999999999e-07 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660443+0000", 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:36.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660450+0000", 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999997e-06 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660454+0000", 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0029999999999996e-06 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679188+0000", 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018734869000000001 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679271+0000", 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2721999999999998e-05 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679499+0000", 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000227612 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101783 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430874807000002, 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019291800000000001, 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101783 2024-09-16T14:08:36.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660458+0000", 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4499999999999997e-07 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660465+0000", 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4659999999999996e-06 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660471+0000", 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5250000000000002e-06 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660480+0000", 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5779999999999999e-06 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679530+0000", 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019050085000000001 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679633+0000", 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103341 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679750+0000", 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011636 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430847538000002, 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.152 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:36.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430830346999997, 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:36.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:36.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.43079333, 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.161 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:36.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.165 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430772654000002, 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:36.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:36.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430730474000001, 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.170 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430696249, 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:36.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430666533, 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.179 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:36.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.430646871, 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:36.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.421886745999998, 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:36.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:36.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.337160204, 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.193 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:08:36.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.197 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:36.198 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:36.199 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:36.196+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:36.299 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:36.296+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:36.305 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:36.306 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:36.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:36.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:08:36.419 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:36.419 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:36.419 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:36.420 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:36.420 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:36.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:36.434 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:36.432+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:36.501 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:36.496+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.700293908999999, 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:08:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:08:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:08:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.700277570000001, 2024-09-16T14:08:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:08:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.700252405000001, 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:08:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.699348438000001, 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:08:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:08:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.699339098999999, 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:08:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:08:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.699048977, 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:08:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:08:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.698830282999999, 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:08:36.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:08:36.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.692436703999999, 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:08:36.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.688525141, 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:08:36.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.688493936, 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:36.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:08:36.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.594 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.687674496, 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:08:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:08:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.687629687000001, 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:08:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.684620432999999, 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:08:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:08:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.684603472999999, 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:08:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.684591977, 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:08:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:08:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.679845066999999, 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:08:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:08:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.672212435999999, 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:08:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:08:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:08:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.672200229000001, 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:08:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.669945314, 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:08:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.669721063000001, 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:08:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:08:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:08:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.650 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:36.651 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone34 from 0xcc0aaa3 (overlap 0xd5acd9) to 0xc04ec3 2024-09-16T14:08:36.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:36.652 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:36.648+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:36.752 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:36.746+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:36.754 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:36.754 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:36.754 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:36.763 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:36.801 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:08:36.801 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:08:36.801 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T14:08:36.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-16T14:08:36.853 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:36.848+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:36.858 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:36.858 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:36.859 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:36.868 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:36.888 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-16T14:08:36.954 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:36.952+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:36.971 INFO:teuthology.orchestra.run.smithi079.stdout:4005 punch from 0x9b1c9f to 0x9b3170, (0x14d1 bytes) 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.598523312000001, 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.978 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:08:36.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.594591894000001, 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.983 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:36.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:08:36.987 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.594567701999999, 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.988 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:08:36.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:36.992 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.590848431000001, 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:08:36.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.586931666000002, 2024-09-16T14:08:36.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:08:36.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:08:37.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.582549387, 2024-09-16T14:08:37.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:08:37.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.582525545999999, 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:08:37.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:08:37.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.582510358, 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.582495772000001, 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:08:37.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:08:37.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.582392122000002, 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:08:37.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.025 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.582364395999999, 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:08:37.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:08:37.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.57969078, 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:08:37.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:08:37.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:08:37.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:08:37.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.579664065999999, 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:08:37.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.579623899000001, 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:08:37.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:08:37.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.579534184, 2024-09-16T14:08:37.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.579125380000001, 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.049 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:08:37.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.579025196, 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:08:37.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:37.058 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.578126558000001, 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:08:37.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.570927825999998, 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.063 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:08:37.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:08:37.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.568759534000002, 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.068 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:08:37.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:08:37.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.073 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:37.075 INFO:teuthology.orchestra.run.smithi079.stdout:4006 read 0x6931105 thru 0x693be94 (0xad90 bytes) 2024-09-16T14:08:37.075 INFO:teuthology.orchestra.run.smithi079.stdout:4007 read 0x41fff7f thru 0x4206759 (0x67db bytes) 2024-09-16T14:08:37.075 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:37.075 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:37.075 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:37.075 INFO:teuthology.orchestra.run.smithi079.stdout:4012 punch from 0x89c9458 to 0x89d2b35, (0x96dd bytes) 2024-09-16T14:08:37.075 INFO:teuthology.orchestra.run.smithi079.stdout:4014 trunc from 0xcc0aaa3 to 0x79d7b92 2024-09-16T14:08:37.075 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:37.072+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:37.076 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:37.085 INFO:teuthology.orchestra.run.smithi079.stdout:4015 write 0xbfc9273 thru 0xbfd32cd (0xa05b bytes) 2024-09-16T14:08:37.091 INFO:teuthology.orchestra.run.smithi079.stdout:4016 trunc from 0xbfd32ce to 0xd8d6c4a 2024-09-16T14:08:37.102 INFO:teuthology.orchestra.run.smithi079.stdout:4019 read 0xb186290 thru 0xb1949dd (0xe74e bytes) 2024-09-16T14:08:37.118 INFO:teuthology.orchestra.run.smithi079.stdout:4020 write 0xecc74fa thru 0xecd4fc2 (0xdac9 bytes) 2024-09-16T14:08:37.125 INFO:teuthology.orchestra.run.smithi079.stdout:4022 read 0xe3d3adc thru 0xe3e292b (0xee50 bytes) 2024-09-16T14:08:37.125 INFO:teuthology.orchestra.run.smithi079.stdout:4023 writesame 0x6a30d39 thru 0x6a3324c data_size 0x2a(0x2514 bytes) 2024-09-16T14:08:37.128 INFO:teuthology.orchestra.run.smithi079.stdout:4024 read 0xddc6768 thru 0xddc8fda (0x2873 bytes) 2024-09-16T14:08:37.129 INFO:teuthology.orchestra.run.smithi079.stdout:4025 clone 36 order 20 su 65536 sc 9 2024-09-16T14:08:37.135 INFO:tasks.ceph.mon.a.smithi079.stderr:2024-09-16T14:08:37.132+0000 7f5ada529700 -1 mon.a@0(leader).osd e91 definitely_dead 0 2024-09-16T14:08:37.176 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:37.170+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:37.189 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:37.189 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:37.189 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:37.189 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:37.198 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:37.278 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:37.270+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:37.300 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:37.300 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:37.300 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:37.309 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:37.378 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:37.376+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:37.417 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:08:37.418 INFO:teuthology.orchestra.run.smithi079.stderr:marked out osd.7. 2024-09-16T14:08:37.461 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:37.461 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:37.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:37.461 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:37.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.461 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:08:37.461 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 76.048554429000006, 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:08:37.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:08:37.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:37.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.210592106, 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.469 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:08:37.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:37.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.683353117999999, 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.475 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.683329886999999, 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:08:37.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:08:37.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.683306940000001, 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:37.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.488 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.683290431, 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.489 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:08:37.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:37.493 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.683272805000001, 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:08:37.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.677357060999999, 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:08:37.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:37.502 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.677339656000001, 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:08:37.506 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.676985290000001, 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.507 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:37.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.511 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.673841431, 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:37.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:08:37.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.673826145, 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:08:37.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.673809735999999, 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:37.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.673734539000002, 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.525 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:08:37.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.673636991999999, 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:08:37.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:08:37.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.673611750999999, 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:08:37.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.673582697000001, 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:08:37.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:08:37.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.670318708, 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.544 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:08:37.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.670300507, 2024-09-16T14:08:37.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:08:37.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.670261328999999, 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:37.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:08:37.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:37.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:37.557 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:37.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:37.559 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:37.550+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:37.659 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:37.658+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:37.665 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:37.665 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:37.665 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:37.665 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:37.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:37.760 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:37.754+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:37.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:37.775 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:37.775 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:37.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:37.861 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:37.854+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:37.886 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:37.886 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:37.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:37.886 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:37.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.646171121, 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:08:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b774ddc9:::rbd_data.1136f7114f14.000000000000007f:18, version: 63'168, data_included: [0~10240,45056~10240,92160~8192,137216~10240,182272~10240,229376~8192], data_size: 57344, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b774ddc9:::rbd_data.1136f7114f14.000000000000007f:18@63'168, size: 237568, copy_subset: [0~237568], clone_subset: {}, snapset: 18=[]:{18=[18]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:237568, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.5193398679999994, 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042357389999999997, 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363380+0000", 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363464+0000", 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4358999999999996e-05 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363496+0000", 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1704e-05 2024-09-16T14:08:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363510+0000", 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.428e-05 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363685+0000", 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017525599999999999 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363709+0000", 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3768999999999999e-05 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.367614+0000", 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003905281 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:146577 3.11 3:8b7c2e51:::rbd_data.113630ceda18.000000000000001c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 751252~2412] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:32.703808+0000", 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.1789108629999996, 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058274629999999997, 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 146577 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.703808+0000", 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.703808+0000", 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.703810+0000", 2024-09-16T14:08:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704517+0000", 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070752900000000004 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704540+0000", 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2357999999999998e-05 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704547+0000", 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1740000000000003e-06 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704631+0000", 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3950999999999996e-05 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704889+0000", 2024-09-16T14:08:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025792600000000001 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.705323+0000", 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00043472399999999999 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.708591+0000", 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032680140000000001 2024-09-16T14:08:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.709565+0000", 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097351800000000004 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.709609+0000", 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4078e-05 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.709635+0000", 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6208999999999999e-05 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:147058 3.a 3:5164f577:::rbd_data.113630ceda18.000000000000005e:head [stat,write 962241~20799] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:32.869273+0000", 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.0134455500000001, 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050651819999999997, 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 147058 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869273+0000", 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869273+0000", 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869275+0000", 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6619999999999999e-06 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869292+0000", 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7340999999999999e-05 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869310+0000", 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8148999999999999e-05 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869318+0000", 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2989999999999999e-06 2024-09-16T14:08:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869392+0000", 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4260999999999999e-05 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869546+0000", 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015466500000000001 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869727+0000", 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018077300000000001 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.870715+0000", 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098812300000000004 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.874221+0000", 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035058429999999998 2024-09-16T14:08:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.874272+0000", 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1088999999999998e-05 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.874338+0000", 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5976999999999995e-05 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:147198 3.d 3:b4e96b78:::rbd_data.113630ceda18.0000000000000053:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:32.951134+0000", 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.931584763, 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042968579999999998, 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 147198 2024-09-16T14:08:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951134+0000", 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951134+0000", 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951134+0000", 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9599999999999999e-07 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951137+0000", 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8420000000000001e-06 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951144+0000", 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4069999999999998e-06 2024-09-16T14:08:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951148+0000", 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3529999999999997e-06 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951648+0000", 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000499848 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951713+0000", 2024-09-16T14:08:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5365999999999997e-05 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951814+0000", 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101005 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.953344+0000", 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001529915 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.955327+0000", 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019828409999999999 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.955387+0000", 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9768000000000001e-05 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.955431+0000", 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4017000000000002e-05 2024-09-16T14:08:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:147200 3.1d 3:bebf3c2f:::rbd_data.113630ceda18.0000000000000055:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:32.956182+0000", 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.9265370649999998, 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047465509999999999, 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 147200 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956182+0000", 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956182+0000", 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956184+0000", 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.587e-06 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956197+0000", 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2425999999999999e-05 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956230+0000", 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3129000000000001e-05 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956242+0000", 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2289999999999999e-05 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956305+0000", 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3068e-05 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956357+0000", 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1533000000000003e-05 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956443+0000", 2024-09-16T14:08:37.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6850999999999999e-05 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.957779+0000", 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013358269999999999 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.960663+0000", 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028833119999999999 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.960807+0000", 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014437 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.960928+0000", 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012115799999999999 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150059 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.922 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.904911+0000", 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97780787400000002, 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040616530000000001, 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150059 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904911+0000", 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904911+0000", 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904911+0000", 2024-09-16T14:08:37.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1999999999999999e-07 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904923+0000", 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1314e-05 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904935+0000", 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2767e-05 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904942+0000", 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4860000000000002e-06 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.908685+0000", 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037434009999999999 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.908827+0000", 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014140299999999999 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.908972+0000", 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014566199999999999 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150109 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905078+0000", 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97764030800000001, 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042129699999999999, 2024-09-16T14:08:37.927 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150109 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905078+0000", 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905078+0000", 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905079+0000", 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1999999999999998e-07 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905082+0000", 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1719999999999999e-06 2024-09-16T14:08:37.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905094+0000", 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1321999999999999e-05 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905100+0000", 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9100000000000002e-06 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909013+0000", 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039129560000000004 2024-09-16T14:08:37.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909148+0000", 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013590499999999999 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909291+0000", 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014278500000000001 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150110 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905107+0000", 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97761155799999999, 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045282430000000004, 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150110 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905107+0000", 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905107+0000", 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905108+0000", 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4000000000000001e-07 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905111+0000", 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1580000000000001e-06 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905124+0000", 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3254999999999999e-05 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905133+0000", 2024-09-16T14:08:37.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9339999999999995e-06 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909331+0000", 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004197742 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909466+0000", 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013534699999999999 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909635+0000", 2024-09-16T14:08:37.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016906700000000001 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150111 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905140+0000", 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97757819899999998, 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046757200000000004, 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150111 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905140+0000", 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905140+0000", 2024-09-16T14:08:37.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905141+0000", 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8000000000000005e-07 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905146+0000", 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905156+0000", 2024-09-16T14:08:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0329999999999999e-05 2024-09-16T14:08:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905162+0000", 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1229999999999997e-06 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909658+0000", 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044955020000000002 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909731+0000", 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3211000000000001e-05 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909816+0000", 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4983999999999998e-05 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150112 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905170+0000", 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97754904600000003, 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048350679999999997, 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150112 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905170+0000", 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905170+0000", 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905170+0000", 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7599999999999996e-07 2024-09-16T14:08:37.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905174+0000", 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.252e-06 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905184+0000", 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0668e-05 2024-09-16T14:08:37.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905190+0000", 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6080000000000003e-06 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909841+0000", 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046515949999999997 2024-09-16T14:08:37.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909914+0000", 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2516999999999997e-05 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910005+0000", 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0651999999999999e-05 2024-09-16T14:08:37.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150114 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905197+0000", 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97752160200000004, 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049960949999999999, 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150114 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905197+0000", 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905197+0000", 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905198+0000", 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6199999999999997e-07 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905201+0000", 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3840000000000001e-06 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905215+0000", 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3805000000000001e-05 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905222+0000", 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6579999999999996e-06 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910026+0000", 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048041919999999997 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910097+0000", 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1286999999999997e-05 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910193+0000", 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6007000000000004e-05 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.952 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150115 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905235+0000", 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97748391300000004, 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051380549999999999, 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150115 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905235+0000", 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905235+0000", 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905236+0000", 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6000000000000002e-07 2024-09-16T14:08:37.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905239+0000", 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2940000000000001e-06 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905250+0000", 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0883999999999999e-05 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905256+0000", 2024-09-16T14:08:37.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3230000000000003e-06 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910214+0000", 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004958339 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910287+0000", 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2302e-05 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910373+0000", 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6052999999999998e-05 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150116 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905263+0000", 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97745565000000001, 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052908959999999998, 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.957 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150116 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905263+0000", 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905263+0000", 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905264+0000", 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1800000000000005e-07 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905273+0000", 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6679999999999996e-06 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905284+0000", 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0564000000000001e-05 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905290+0000", 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2319999999999998e-06 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910395+0000", 2024-09-16T14:08:37.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051043540000000002 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910467+0000", 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2805999999999994e-05 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910554+0000", 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6553999999999998e-05 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.961 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150117 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905296+0000", 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.97742221100000004, 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054373299999999998, 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150117 2024-09-16T14:08:37.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905296+0000", 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905296+0000", 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905297+0000", 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7599999999999997e-07 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905310+0000", 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2629999999999999e-05 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905322+0000", 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1905999999999999e-05 2024-09-16T14:08:37.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905329+0000", 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0079999999999996e-06 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910575+0000", 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052461970000000002 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910647+0000", 2024-09-16T14:08:37.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2115999999999999e-05 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910734+0000", 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6797000000000003e-05 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150288 3.13 3:cf7ed1c6:::rbd_data.113673b9891.0000000000000004:24 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.966 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.923015+0000", 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.95970360700000001, 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039418830000000002, 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150288 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923015+0000", 2024-09-16T14:08:37.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923015+0000", 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923015+0000", 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923017+0000", 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9190000000000002e-06 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923023+0000", 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6590000000000004e-06 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923027+0000", 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7790000000000002e-06 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.926773+0000", 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003745937 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.926855+0000", 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1928000000000006e-05 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.926957+0000", 2024-09-16T14:08:37.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010223899999999999 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150289 3.13 3:cf7ed1c6:::rbd_data.113673b9891.0000000000000004:24 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.923031+0000", 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.95968764799999995, 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042364489999999998, 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150289 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923031+0000", 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923031+0000", 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923031+0000", 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923033+0000", 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.801e-06 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923039+0000", 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8969999999999996e-06 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923042+0000", 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4209999999999999e-06 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.926984+0000", 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039410679999999998 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927125+0000", 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014192999999999999 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927267+0000", 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141965 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150320 3.13 3:cf7ed1c6:::rbd_data.113673b9891.0000000000000004:24 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.923115+0000", 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.95960336199999996, 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044696370000000003, 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150320 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923115+0000", 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923115+0000", 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923116+0000", 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.08e-07 2024-09-16T14:08:37.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923118+0000", 2024-09-16T14:08:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8649999999999999e-06 2024-09-16T14:08:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923123+0000", 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4500000000000003e-06 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923126+0000", 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3419999999999999e-06 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927336+0000", 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042096800000000004 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927468+0000", 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000131994 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927585+0000", 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116898 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150321 3.13 3:cf7ed1c6:::rbd_data.113673b9891.0000000000000004:24 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.923130+0000", 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.95958851300000003, 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045837350000000002, 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150321 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923130+0000", 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923130+0000", 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923131+0000", 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0900000000000002e-07 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923138+0000", 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.374e-06 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923144+0000", 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0059999999999998e-06 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923148+0000", 2024-09-16T14:08:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6629999999999999e-06 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927604+0000", 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044568749999999999 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927656+0000", 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1591999999999997e-05 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927714+0000", 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7815999999999997e-05 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:150345 3.16 e91/88 3:6ada7db2:::rbd_data.113630ceda18.0000000000000004:head v 91'421, mlcod=89'418)", 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.932998+0000", 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.94972065000000006, 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042299909999999998, 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.932998+0000", 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.932998+0000", 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.932999+0000", 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.299e-06 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933545+0000", 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054559300000000003 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933568+0000", 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2955999999999998e-05 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933577+0000", 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.901e-06 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933618+0000", 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1051999999999999e-05 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933636+0000", 2024-09-16T14:08:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8091999999999999e-05 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.937182+0000", 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003546338 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.937228+0000", 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5760000000000002e-05 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:37.991 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:37.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:37.993 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:37.993 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:37.993 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:37.994 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:37.994 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:37.986+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:37.999 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:38.022 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:38.045 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:38.067 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:38.069 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:38.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:38.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:38.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:38.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:38.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:38.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:38.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:38.093 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:38.086+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:38.097 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:38.097 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:38.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:38.194 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:38.186+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:38.205 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:38.206 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:38.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:38.295 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:38.290+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:38.312 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:38.312 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:38.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:38.396 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:38.390+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:38.431 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:38.432 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:38.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:38.497 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:38.494+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:38.535 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:38.535 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:38.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:38.597 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:38.590+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:38.640 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:38.640 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:38.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:38.697 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:38.692+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:38.750 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:38.750 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:38.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:38.799 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:38.796+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:38.854 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:38.854 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:38.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:38.906 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:38.904+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:39.000 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:38.996+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:39.007 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.007 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.016 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:39.100 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:39.094+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:39.123 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.124 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.133 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:39.201 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:39.196+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:39.232 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.232 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.241 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:39.302 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:39.294+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:39.346 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.346 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.355 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:39.403 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:39.400+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:39.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:08:39.461 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.461 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:39.503 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:39.500+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:39.555 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone35 from 0xecd4fc3 (overlap 0x79d7b92) to 0x0 2024-09-16T14:08:39.588 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.588 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:39.604 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:39.598+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:39.705 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:39.700+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:39.706 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.706 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.717 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:39.805 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:39.800+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:39.826 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.826 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.833 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-16T14:08:39.836 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:39.899 INFO:teuthology.orchestra.run.smithi079.stdout:4026 trunc from 0xecd4fc3 to 0x78427b0 2024-09-16T14:08:39.907 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:39.902+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:39.933 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:39.933 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:39.936 INFO:teuthology.orchestra.run.smithi079.stdout:4028 writesame 0x2350dfc thru 0x235fb81 data_size 0x1(0xed86 bytes) 2024-09-16T14:08:39.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:39.975 INFO:teuthology.orchestra.run.smithi079.stdout:4029 read 0x2807ba8 thru 0x2809f16 (0x236f bytes) 2024-09-16T14:08:39.976 INFO:teuthology.orchestra.run.smithi079.stdout:4030 read 0x6470268 thru 0x6474595 (0x432e bytes) 2024-09-16T14:08:39.978 INFO:teuthology.orchestra.run.smithi079.stdout:4031 write 0x50e69ce thru 0x50f1627 (0xac5a bytes) 2024-09-16T14:08:39.988 INFO:teuthology.orchestra.run.smithi079.stdout:4032 writesame 0x8432af thru 0x84d615 data_size 0x1(0xa367 bytes) 2024-09-16T14:08:40.007 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:40.002+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:40.028 INFO:teuthology.orchestra.run.smithi079.stdout:4033 writesame 0xac5bcea thru 0xac5d73d data_size 0xa(0x1a54 bytes) 2024-09-16T14:08:40.029 INFO:teuthology.orchestra.run.smithi079.stdout:4034 write 0x6d56639 thru 0x6d61e60 (0xb828 bytes) 2024-09-16T14:08:40.042 INFO:teuthology.orchestra.run.smithi079.stdout:4035 read 0x4168d49 thru 0x416b133 (0x23eb bytes) 2024-09-16T14:08:40.042 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:40.042 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:40.043 INFO:teuthology.orchestra.run.smithi079.stdout:4036 punch from 0x736e515 to 0x7376fa8, (0x8a93 bytes) 2024-09-16T14:08:40.043 INFO:teuthology.orchestra.run.smithi079.stdout:4037 writesame 0x249de8b thru 0x24a1b12 data_size 0x1a(0x3c88 bytes) 2024-09-16T14:08:40.051 INFO:teuthology.orchestra.run.smithi079.stdout:4038 read 0xa59a2ce thru 0xa5a95bf (0xf2f2 bytes) 2024-09-16T14:08:40.051 INFO:teuthology.orchestra.run.smithi079.stdout:4039 write 0x263900d thru 0x264618f (0xd183 bytes) 2024-09-16T14:08:40.053 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:40.059 INFO:teuthology.orchestra.run.smithi079.stdout:4041 write 0xa7ba301 thru 0xa7be016 (0x3d16 bytes) 2024-09-16T14:08:40.064 INFO:teuthology.orchestra.run.smithi079.stdout:4042 read 0x9809039 thru 0x9812c8c (0x9c54 bytes) 2024-09-16T14:08:40.064 INFO:teuthology.orchestra.run.smithi079.stdout:4043 read 0x7eb16a6 thru 0x7ebc805 (0xb160 bytes) 2024-09-16T14:08:40.064 INFO:teuthology.orchestra.run.smithi079.stdout:4044 write 0x893518c thru 0x894363a (0xe4af bytes) 2024-09-16T14:08:40.072 INFO:teuthology.orchestra.run.smithi079.stdout:4045 flatten 2024-09-16T14:08:40.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:40.126 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:40.124+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:40.186 INFO:teuthology.orchestra.run.smithi079.stdout:4046 read 0xab882ce thru 0xab8c635 (0x4368 bytes) 2024-09-16T14:08:40.186 INFO:teuthology.orchestra.run.smithi079.stdout:4048 write 0x1d6cff thru 0x1db0e8 (0x43ea bytes) 2024-09-16T14:08:40.191 INFO:teuthology.orchestra.run.smithi079.stdout:4049 trunc from 0xac5d73e to 0x8cbf49a 2024-09-16T14:08:40.208 INFO:teuthology.orchestra.run.smithi079.stdout:4050 read 0x2b6a2fa thru 0x2b74da9 (0xaab0 bytes) 2024-09-16T14:08:40.209 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:40.209 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:40.209 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:40.204+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:40.210 INFO:teuthology.orchestra.run.smithi079.stdout:4052 read 0x8c15921 thru 0x8c174f1 (0x1bd1 bytes) 2024-09-16T14:08:40.210 INFO:teuthology.orchestra.run.smithi079.stdout:4053 trunc from 0x8cbf49a to 0x8a62cb7 2024-09-16T14:08:40.219 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:40.220 INFO:teuthology.orchestra.run.smithi079.stdout:4054 writesame 0x23c3453 thru 0x23ce9c6 data_size 0x62(0xb574 bytes) 2024-09-16T14:08:40.225 INFO:teuthology.orchestra.run.smithi079.stdout:4056 punch from 0x40a5548 to 0x40a6528, (0xfe0 bytes) 2024-09-16T14:08:40.225 INFO:teuthology.orchestra.run.smithi079.stdout:4057 read 0x4c2a605 thru 0x4c2d65c (0x3058 bytes) 2024-09-16T14:08:40.225 INFO:teuthology.orchestra.run.smithi079.stdout:4058 read 0x49ce745 thru 0x49dc6dd (0xdf99 bytes) 2024-09-16T14:08:40.225 INFO:teuthology.orchestra.run.smithi079.stdout:4062 write 0x8939421 thru 0x8941b83 (0x8763 bytes) 2024-09-16T14:08:40.229 INFO:teuthology.orchestra.run.smithi079.stdout:4063 write 0x824dfba thru 0x825b8f9 (0xd940 bytes) 2024-09-16T14:08:40.236 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:40.237 INFO:teuthology.orchestra.run.smithi079.stdout:4064 write 0x43e585c thru 0x43e9ef6 (0x469b bytes) 2024-09-16T14:08:40.254 INFO:teuthology.orchestra.run.smithi079.stdout:4065 write 0xb4bd7ae thru 0xb4c13db (0x3c2e bytes) 2024-09-16T14:08:40.260 INFO:teuthology.orchestra.run.smithi079.stdout:4066 punch from 0x2c4a3e7 to 0x2c4ab09, (0x722 bytes) 2024-09-16T14:08:40.263 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:40.270 INFO:teuthology.orchestra.run.smithi079.stdout:4067 writesame 0xbfcc10b thru 0xbfd4c99 data_size 0x1(0x8b8f bytes) 2024-09-16T14:08:40.287 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:40.289 INFO:teuthology.orchestra.run.smithi079.stdout:4068 trunc from 0xbfd4c9a to 0x89c507 2024-09-16T14:08:40.309 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:40.302+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:40.314 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:40.326 INFO:teuthology.orchestra.run.smithi079.stdout:4069 write 0xd56795 thru 0xd5fb0a (0x9376 bytes) 2024-09-16T14:08:40.326 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:40.326 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:40.331 INFO:teuthology.orchestra.run.smithi079.stdout:4070 punch from 0x8a775b to 0x8b1270, (0x9b15 bytes) 2024-09-16T14:08:40.331 INFO:teuthology.orchestra.run.smithi079.stdout:4071 punch from 0x93a367 to 0x9472bf, (0xcf58 bytes) 2024-09-16T14:08:40.334 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:40.336 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:40.347 INFO:teuthology.orchestra.run.smithi079.stdout:4076 trunc from 0xd5fb0b to 0x9eb66bd 2024-09-16T14:08:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:08:40.361 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:4080 read 0x38baeb4 thru 0x38c522c (0xa379 bytes) 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:4081 read 0x7a6e14f thru 0x7a78588 (0xa43a bytes) 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:4083 read 0x5e8b4e3 thru 0x5e8ceac (0x19ca bytes) 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:4086 punch from 0x7a69b69 to 0x7a6b44c, (0x18e3 bytes) 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:4087 punch from 0x148f80a to 0x149662c, (0x6e22 bytes) 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:4088 punch from 0x263abd6 to 0x2640af8, (0x5f22 bytes) 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:4089 read 0x93c2bc6 thru 0x93c4917 (0x1d52 bytes) 2024-09-16T14:08:40.362 INFO:teuthology.orchestra.run.smithi079.stdout:4091 write 0x5d623cc thru 0x5d68c25 (0x685a bytes) 2024-09-16T14:08:40.363 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:40.370 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:40.370 INFO:teuthology.orchestra.run.smithi079.stdout:4093 write 0x846c547 thru 0x8475fd3 (0x9a8d bytes) 2024-09-16T14:08:40.376 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:40.376 INFO:teuthology.orchestra.run.smithi079.stdout:4096 read 0xc2e1a1 thru 0xc3e02e (0xfe8e bytes) 2024-09-16T14:08:40.386 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:40.392 INFO:teuthology.orchestra.run.smithi079.stdout:4097 trunc from 0x9eb66bd to 0x26bfa7 2024-09-16T14:08:40.408 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:40.410 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:40.410 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:40.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:40.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:40.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:40.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:40.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:40.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:40.412 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:40.406+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:40.439 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:08:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:40.439 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.025659536999996, 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:40.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:08:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:08:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:08:40.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:08:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:08:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:08:40.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:08:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:08:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:08:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:08:40.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.187697214000003, 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:40.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:08:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.660458225999999, 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:08:40.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.459 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.660434994999999, 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:08:40.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:08:40.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:08:40.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.465 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.660412048000001, 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:08:40.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:08:40.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.660395539, 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:08:40.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:08:40.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:08:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:08:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:08:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:08:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:08:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:08:40.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:08:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:08:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:08:40.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:08:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:08:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.479 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.480 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.660377913000001, 2024-09-16T14:08:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:08:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:08:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:08:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:08:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:08:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:08:40.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:08:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:08:40.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:08:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:08:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:08:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:08:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:08:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:08:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:08:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.654462168999999, 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:40.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:08:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:08:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.654444764000001, 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:08:40.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:08:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.497 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:08:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:08:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:08:40.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:08:40.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:40.503 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.654090398000001, 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:40.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:08:40.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.650946539, 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:40.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:08:40.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:08:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:08:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:08:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:08:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:08:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:08:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:08:40.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.650931253, 2024-09-16T14:08:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:08:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:08:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:40.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:08:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:08:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:08:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:08:40.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:08:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:08:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:08:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:08:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:08:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:08:40.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.650914843999999, 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:08:40.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.526 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.650839647000002, 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.650742099999999, 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:08:40.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.650716858999999, 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:40.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.650687805, 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:08:40.541 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:08:40.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.647423816, 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:08:40.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.551 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.647405615, 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:08:40.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:08:40.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.647366436999999, 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:40.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:40.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:08:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:08:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:40.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:08:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:08:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:40.561 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:40.561 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:40.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:40.563 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:40.560+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:40.664 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:40.660+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:40.668 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:40.669 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:40.669 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:40.669 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:40.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:40.764 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:40.760+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:40.778 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:40.778 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:40.778 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:40.778 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:40.778 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:40.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:40.866 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:40.864+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:40.934 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.693991717999999, 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:08:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGPush(3.d 84/83 [PushOp(3:b774ddc9:::rbd_data.1136f7114f14.000000000000007f:18, version: 63'168, data_included: [0~10240,45056~10240,92160~8192,137216~10240,182272~10240,229376~8192], data_size: 57344, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:b774ddc9:::rbd_data.1136f7114f14.000000000000007f:18@63'168, size: 237568, copy_subset: [0~237568], clone_subset: {}, snapset: 18=[]:{18=[18]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:237568, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.567160465000001, 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042357389999999997, 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363379+0000", 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363380+0000", 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363464+0000", 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4358999999999996e-05 2024-09-16T14:08:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363496+0000", 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1704e-05 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363510+0000", 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.428e-05 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363685+0000", 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017525599999999999 2024-09-16T14:08:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.363709+0000", 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3768999999999999e-05 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.367614+0000", 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003905281 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:146577 3.11 3:8b7c2e51:::rbd_data.113630ceda18.000000000000001c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 751252~2412] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:32.703808+0000", 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.2267314599999999, 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058274629999999997, 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 146577 2024-09-16T14:08:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.703808+0000", 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.703808+0000", 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.703810+0000", 2024-09-16T14:08:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704517+0000", 2024-09-16T14:08:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00070752900000000004 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704540+0000", 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2357999999999998e-05 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704547+0000", 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1740000000000003e-06 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704631+0000", 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3950999999999996e-05 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.704889+0000", 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025792600000000001 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.705323+0000", 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00043472399999999999 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.708591+0000", 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032680140000000001 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.709565+0000", 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00097351800000000004 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.709609+0000", 2024-09-16T14:08:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4078e-05 2024-09-16T14:08:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.709635+0000", 2024-09-16T14:08:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6208999999999999e-05 2024-09-16T14:08:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:147058 3.a 3:5164f577:::rbd_data.113630ceda18.000000000000005e:head [stat,write 962241~20799] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:32.869273+0000", 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.0612661469999995, 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050651819999999997, 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 147058 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869273+0000", 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869273+0000", 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869275+0000", 2024-09-16T14:08:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6619999999999999e-06 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869292+0000", 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7340999999999999e-05 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869310+0000", 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8148999999999999e-05 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869318+0000", 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2989999999999999e-06 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869392+0000", 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4260999999999999e-05 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869546+0000", 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015466500000000001 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.869727+0000", 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018077300000000001 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.870715+0000", 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098812300000000004 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.874221+0000", 2024-09-16T14:08:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035058429999999998 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.874272+0000", 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1088999999999998e-05 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.874338+0000", 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5976999999999995e-05 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:147198 3.d 3:b4e96b78:::rbd_data.113630ceda18.0000000000000053:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:32.951134+0000", 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9794053600000003, 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042968579999999998, 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 147198 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951134+0000", 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951134+0000", 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951134+0000", 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9599999999999999e-07 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951137+0000", 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8420000000000001e-06 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951144+0000", 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4069999999999998e-06 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951148+0000", 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3529999999999997e-06 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951648+0000", 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000499848 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951713+0000", 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5365999999999997e-05 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.951814+0000", 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101005 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.953344+0000", 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001529915 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.955327+0000", 2024-09-16T14:08:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019828409999999999 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.955387+0000", 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9768000000000001e-05 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.955431+0000", 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4017000000000002e-05 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:147200 3.1d 3:bebf3c2f:::rbd_data.113630ceda18.0000000000000055:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:32.956182+0000", 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.9743576620000001, 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047465509999999999, 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 147200 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956182+0000", 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956182+0000", 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956184+0000", 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.587e-06 2024-09-16T14:08:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956197+0000", 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2425999999999999e-05 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956230+0000", 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3129000000000001e-05 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956242+0000", 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2289999999999999e-05 2024-09-16T14:08:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956305+0000", 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3068e-05 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956357+0000", 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1533000000000003e-05 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.956443+0000", 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6850999999999999e-05 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.957779+0000", 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013358269999999999 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.960663+0000", 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028833119999999999 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.960807+0000", 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014437 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.960928+0000", 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012115799999999999 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150059 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.904911+0000", 2024-09-16T14:08:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0256284710000001, 2024-09-16T14:08:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040616530000000001, 2024-09-16T14:08:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150059 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904911+0000", 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904911+0000", 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904911+0000", 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1999999999999999e-07 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904923+0000", 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1314e-05 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904935+0000", 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2767e-05 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.904942+0000", 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4860000000000002e-06 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.908685+0000", 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037434009999999999 2024-09-16T14:08:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.908827+0000", 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014140299999999999 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.908972+0000", 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014566199999999999 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150109 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905078+0000", 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0254609050000001, 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042129699999999999, 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150109 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905078+0000", 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905078+0000", 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905079+0000", 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1999999999999998e-07 2024-09-16T14:08:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905082+0000", 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1719999999999999e-06 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905094+0000", 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1321999999999999e-05 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905100+0000", 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9100000000000002e-06 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909013+0000", 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039129560000000004 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909148+0000", 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013590499999999999 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909291+0000", 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014278500000000001 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150110 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905107+0000", 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0254321549999998, 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045282430000000004, 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150110 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905107+0000", 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905107+0000", 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905108+0000", 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4000000000000001e-07 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905111+0000", 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1580000000000001e-06 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905124+0000", 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3254999999999999e-05 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905133+0000", 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9339999999999995e-06 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909331+0000", 2024-09-16T14:08:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004197742 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909466+0000", 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013534699999999999 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909635+0000", 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016906700000000001 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150111 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905140+0000", 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0253987960000002, 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046757200000000004, 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150111 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905140+0000", 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905140+0000", 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905141+0000", 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8000000000000005e-07 2024-09-16T14:08:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905146+0000", 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7899999999999999e-06 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905156+0000", 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0329999999999999e-05 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905162+0000", 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1229999999999997e-06 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909658+0000", 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044955020000000002 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909731+0000", 2024-09-16T14:08:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3211000000000001e-05 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909816+0000", 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4983999999999998e-05 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150112 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905170+0000", 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0253696430000003, 2024-09-16T14:08:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048350679999999997, 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150112 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905170+0000", 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905170+0000", 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905170+0000", 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7599999999999996e-07 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905174+0000", 2024-09-16T14:08:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.252e-06 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905184+0000", 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0668e-05 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905190+0000", 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6080000000000003e-06 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909841+0000", 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046515949999999997 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.909914+0000", 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2516999999999997e-05 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910005+0000", 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0651999999999999e-05 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150114 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905197+0000", 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0253421989999998, 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049960949999999999, 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150114 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905197+0000", 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905197+0000", 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905198+0000", 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6199999999999997e-07 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905201+0000", 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3840000000000001e-06 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905215+0000", 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3805000000000001e-05 2024-09-16T14:08:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905222+0000", 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6579999999999996e-06 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910026+0000", 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048041919999999997 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910097+0000", 2024-09-16T14:08:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1286999999999997e-05 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910193+0000", 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6007000000000004e-05 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150115 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 983040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905235+0000", 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0253045099999998, 2024-09-16T14:08:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051380549999999999, 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150115 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905235+0000", 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905235+0000", 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905236+0000", 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6000000000000002e-07 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905239+0000", 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2940000000000001e-06 2024-09-16T14:08:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905250+0000", 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0883999999999999e-05 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905256+0000", 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3230000000000003e-06 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910214+0000", 2024-09-16T14:08:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004958339 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910287+0000", 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2302e-05 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910373+0000", 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6052999999999998e-05 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150116 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905263+0000", 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0252762469999999, 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052908959999999998, 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150116 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905263+0000", 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905263+0000", 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905264+0000", 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1800000000000005e-07 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905273+0000", 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6679999999999996e-06 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905284+0000", 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0564000000000001e-05 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905290+0000", 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2319999999999998e-06 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910395+0000", 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051043540000000002 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910467+0000", 2024-09-16T14:08:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2805999999999994e-05 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910554+0000", 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6553999999999998e-05 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150117 3.14 3:297a281c:::rbd_data.113673b9891.0000000000000009:24 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.905296+0000", 2024-09-16T14:08:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0252428079999998, 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054373299999999998, 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150117 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905296+0000", 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905296+0000", 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905297+0000", 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7599999999999997e-07 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905310+0000", 2024-09-16T14:08:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2629999999999999e-05 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905322+0000", 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1905999999999999e-05 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.905329+0000", 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0079999999999996e-06 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910575+0000", 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0052461970000000002 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910647+0000", 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2115999999999999e-05 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.910734+0000", 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6797000000000003e-05 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150288 3.13 3:cf7ed1c6:::rbd_data.113673b9891.0000000000000004:24 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.923015+0000", 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0075242040000001, 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039418830000000002, 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150288 2024-09-16T14:08:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923015+0000", 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923015+0000", 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923015+0000", 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2199999999999999e-07 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923017+0000", 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9190000000000002e-06 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923023+0000", 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6590000000000004e-06 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923027+0000", 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7790000000000002e-06 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.926773+0000", 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003745937 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.926855+0000", 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1928000000000006e-05 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.926957+0000", 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010223899999999999 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150289 3.13 3:cf7ed1c6:::rbd_data.113673b9891.0000000000000004:24 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.923031+0000", 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0075082450000004, 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042364489999999998, 2024-09-16T14:08:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150289 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923031+0000", 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923031+0000", 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923031+0000", 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6699999999999999e-07 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923033+0000", 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.801e-06 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923039+0000", 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8969999999999996e-06 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923042+0000", 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4209999999999999e-06 2024-09-16T14:08:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.926984+0000", 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039410679999999998 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927125+0000", 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014192999999999999 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927267+0000", 2024-09-16T14:08:41.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000141965 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150320 3.13 3:cf7ed1c6:::rbd_data.113673b9891.0000000000000004:24 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.923115+0000", 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0074239589999996, 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044696370000000003, 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150320 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923115+0000", 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923115+0000", 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923116+0000", 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.08e-07 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923118+0000", 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8649999999999999e-06 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923123+0000", 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4500000000000003e-06 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923126+0000", 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3419999999999999e-06 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927336+0000", 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042096800000000004 2024-09-16T14:08:41.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927468+0000", 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000131994 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927585+0000", 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116898 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:150321 3.13 3:cf7ed1c6:::rbd_data.113673b9891.0000000000000004:24 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.923130+0000", 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0074091100000002, 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045837350000000002, 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 150321 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923130+0000", 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923130+0000", 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923131+0000", 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0900000000000002e-07 2024-09-16T14:08:41.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923138+0000", 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.374e-06 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923144+0000", 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0059999999999998e-06 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.923148+0000", 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6629999999999999e-06 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927604+0000", 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044568749999999999 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927656+0000", 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1591999999999997e-05 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.927714+0000", 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7815999999999997e-05 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:150345 3.16 e91/88 3:6ada7db2:::rbd_data.113630ceda18.0000000000000004:head v 91'421, mlcod=89'418)", 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:36.932998+0000", 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.997541247, 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0042299909999999998, 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:41.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.932998+0000", 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.932998+0000", 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.932999+0000", 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.299e-06 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933545+0000", 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00054559300000000003 2024-09-16T14:08:41.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933568+0000", 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2955999999999998e-05 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933577+0000", 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.901e-06 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933618+0000", 2024-09-16T14:08:41.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1051999999999999e-05 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.933636+0000", 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8091999999999999e-05 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.937182+0000", 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003546338 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:36.937228+0000", 2024-09-16T14:08:41.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5760000000000002e-05 2024-09-16T14:08:41.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.300 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:41.301 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:41.300+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:41.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:41.386 INFO:teuthology.orchestra.run.smithi079.stdout:4100 trunc from 0x26bfa7 to 0x549fb3f 2024-09-16T14:08:41.394 INFO:teuthology.orchestra.run.smithi079.stdout:4101 punch from 0x33646a to 0x345f77, (0xfb0d bytes) 2024-09-16T14:08:41.402 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:41.400+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:41.403 INFO:teuthology.orchestra.run.smithi079.stdout:4104 write 0x5b77aa0 thru 0x5b7a4d9 (0x2a3a bytes) 2024-09-16T14:08:41.408 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:41.418 INFO:teuthology.orchestra.run.smithi079.stdout:4106 write 0x5c123bd thru 0x5c166a0 (0x42e4 bytes) 2024-09-16T14:08:41.425 INFO:teuthology.orchestra.run.smithi079.stdout:4107 punch from 0x4f8de1b to 0x4f9618c, (0x8371 bytes) 2024-09-16T14:08:41.425 INFO:teuthology.orchestra.run.smithi079.stdout:4108 read 0x1dbb779 thru 0x1dbc2d8 (0xb60 bytes) 2024-09-16T14:08:41.428 INFO:teuthology.orchestra.run.smithi079.stdout:4111 trunc from 0x5c166a1 to 0x42bdd0d 2024-09-16T14:08:41.460 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:41.460 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:41.460 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:41.460 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:41.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:41.468 INFO:teuthology.orchestra.run.smithi079.stdout:4113 write 0xbb4694f thru 0xbb55e6d (0xf51f bytes) 2024-09-16T14:08:41.476 INFO:teuthology.orchestra.run.smithi079.stdout:4114 read 0xa7dd4d6 thru 0xa7eafd3 (0xdafe bytes) 2024-09-16T14:08:41.476 INFO:teuthology.orchestra.run.smithi079.stdout:4115 read 0x7aba1e1 thru 0x7ac4480 (0xa2a0 bytes) 2024-09-16T14:08:41.476 INFO:teuthology.orchestra.run.smithi079.stdout:4117 write 0x8645e4e thru 0x864a5c1 (0x4774 bytes) 2024-09-16T14:08:41.482 INFO:teuthology.orchestra.run.smithi079.stdout:4118 write 0xb04cf6a thru 0xb051de7 (0x4e7e bytes) 2024-09-16T14:08:41.489 INFO:teuthology.orchestra.run.smithi079.stdout:4120 write 0x689a4b4 thru 0x68a5239 (0xad86 bytes) 2024-09-16T14:08:41.503 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:41.500+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:4122 writesame 0xe9c4f7c thru 0xe9d1122 data_size 0x19(0xc1a7 bytes) 2024-09-16T14:08:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:4123 read 0x118ed04 thru 0x119288f (0x3b8c bytes) 2024-09-16T14:08:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:4126 read 0x6166db7 thru 0x616c2f9 (0x5543 bytes) 2024-09-16T14:08:41.513 INFO:teuthology.orchestra.run.smithi079.stdout:4128 write 0x5e3321 thru 0x5ee4c4 (0xb1a4 bytes) 2024-09-16T14:08:41.517 INFO:teuthology.orchestra.run.smithi079.stdout:4129 write 0xca39d thru 0xd6244 (0xbea8 bytes) 2024-09-16T14:08:41.520 INFO:teuthology.orchestra.run.smithi079.stdout:4130 write 0x59e9938 thru 0x59f4fd6 (0xb69f bytes) 2024-09-16T14:08:41.563 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:41.564 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:41.564 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:41.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:41.604 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:41.598+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:41.706 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:41.704+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:41.716 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:41.716 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:41.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:41.716 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:41.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.716 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.716 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:41.716 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802386953999999, 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:41.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:41.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802377227000001, 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.722 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:41.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:41.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:41.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802353664000002, 2024-09-16T14:08:41.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:41.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:41.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802326424999997, 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:41.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:41.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802317012000003, 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:41.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:41.744 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802292692999998, 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:41.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:41.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802273087000003, 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:41.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:41.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802252899000003, 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:41.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802229629999999, 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:41.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:41.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:41.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802199379999998, 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:41.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:41.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:41.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802182627999997, 2024-09-16T14:08:41.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.778 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802165363999997, 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:41.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:41.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802126528000002, 2024-09-16T14:08:41.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:41.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802091699999998, 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:41.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802055813999999, 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:41.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802033221999999, 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:41.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:41.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.802004150000002, 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:41.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.801985803999997, 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:41.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:41.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.801968465000002, 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:41.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:41.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:41.821 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.801952542000002, 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:41.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:41.826 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:41.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:41.828 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:41.822+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:41.929 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:41.924+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:41.934 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:41.934 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:41.934 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:41.934 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:41.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:42.030 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:42.022+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:42.044 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:42.044 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:42.044 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:42.045 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:42.045 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:42.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:42.130 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:42.128+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:42.157 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:42.157 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.250993786999999, 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:08:42.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:08:42.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.162 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.250971079000003, 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:08:42.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.250946927000001, 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:08:42.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:42.173 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.250913986, 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:08:42.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:08:42.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.250889303000001, 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.179 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:08:42.183 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.250866074000001, 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.184 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:08:42.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.249020109999996, 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:08:42.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.248977246000003, 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:42.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:08:42.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247517627999997, 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.200 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:08:42.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247501859000003, 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:42.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:08:42.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.210 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247486107, 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:42.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:08:42.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247470675999999, 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:08:42.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247455037999998, 2024-09-16T14:08:42.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:42.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:08:42.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.226 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247439899, 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:08:42.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:08:42.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247425061999998, 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:08:42.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:08:42.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:08:42.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247402057999999, 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.237 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:08:42.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:08:42.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247386833, 2024-09-16T14:08:42.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:08:42.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.247 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247371649000002, 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:08:42.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:08:42.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247356404000001, 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:08:42.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.257 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 48.247341245999998, 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:08:42.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:42.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:08:42.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:08:42.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:42.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:42.263 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:42.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:42.265 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:42.262+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:42.365 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:42.360+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:42.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:08:42.375 INFO:teuthology.orchestra.run.smithi079.stdout:4131 write 0xbb7d28f thru 0xbb7de87 (0xbf9 bytes) 2024-09-16T14:08:42.386 INFO:teuthology.orchestra.run.smithi079.stdout:4132 write 0x95f4e46 thru 0x95f8d54 (0x3f0f bytes) 2024-09-16T14:08:42.394 INFO:teuthology.orchestra.run.smithi079.stdout:4133 writesame 0xe17eaec thru 0xe184918 data_size 0x1(0x5e2d bytes) 2024-09-16T14:08:42.415 INFO:teuthology.orchestra.run.smithi079.stdout:4134 punch from 0x50a1649 to 0x50ac3c6, (0xad7d bytes) 2024-09-16T14:08:42.415 INFO:teuthology.orchestra.run.smithi079.stdout:4136 punch from 0x7ac5747 to 0x7ad0cbc, (0xb575 bytes) 2024-09-16T14:08:42.415 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.415 INFO:teuthology.orchestra.run.smithi079.stdout:4138 write 0x76944cb thru 0x76a113a (0xcc70 bytes) 2024-09-16T14:08:42.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:42.424 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:08:42.424 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:08:42.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:08:42.428 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:42.428 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:42.429 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:42.429 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:42.436 INFO:teuthology.orchestra.run.smithi079.stdout:4139 trunc from 0xe9d1123 to 0x5fba0f7 2024-09-16T14:08:42.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:42.457 INFO:teuthology.orchestra.run.smithi079.stdout:4140 read 0x3591ba0 thru 0x359fae9 (0xdf4a bytes) 2024-09-16T14:08:42.457 INFO:teuthology.orchestra.run.smithi079.stdout:4141 read 0x1cc6534 thru 0x1ccdffc (0x7ac9 bytes) 2024-09-16T14:08:42.466 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:42.458+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:42.473 INFO:teuthology.orchestra.run.smithi079.stdout:4142 write 0xb91e11e thru 0xb9287c5 (0xa6a8 bytes) 2024-09-16T14:08:42.482 INFO:teuthology.orchestra.run.smithi079.stdout:4144 read 0x841814f thru 0x8426170 (0xe022 bytes) 2024-09-16T14:08:42.482 INFO:teuthology.orchestra.run.smithi079.stdout:4146 read 0x143ef93 thru 0x144ceeb (0xdf59 bytes) 2024-09-16T14:08:42.482 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.482 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.482 INFO:teuthology.orchestra.run.smithi079.stdout:4150 read 0xa3e1777 thru 0xa3e8d65 (0x75ef bytes) 2024-09-16T14:08:42.482 INFO:teuthology.orchestra.run.smithi079.stdout:4151 read 0x1703d06 thru 0x1705ed0 (0x21cb bytes) 2024-09-16T14:08:42.495 INFO:teuthology.orchestra.run.smithi079.stdout:4152 trunc from 0xb9287c6 to 0x2e69fd6 2024-09-16T14:08:42.511 INFO:teuthology.orchestra.run.smithi079.stdout:4153 punch from 0x1b3deee to 0x1b4bdf2, (0xdf04 bytes) 2024-09-16T14:08:42.511 INFO:teuthology.orchestra.run.smithi079.stdout:4154 read 0x820903 thru 0x8294ec (0x8bea bytes) 2024-09-16T14:08:42.512 INFO:teuthology.orchestra.run.smithi079.stdout:4156 punch from 0x286fce3 to 0x2875371, (0x568e bytes) 2024-09-16T14:08:42.512 INFO:teuthology.orchestra.run.smithi079.stdout:4157 read 0x191b26b thru 0x1922a0b (0x77a1 bytes) 2024-09-16T14:08:42.532 INFO:teuthology.orchestra.run.smithi079.stdout:4158 write 0xa063f98 thru 0xa068287 (0x42f0 bytes) 2024-09-16T14:08:42.538 INFO:teuthology.orchestra.run.smithi079.stdout:4159 punch from 0x30b13d8 to 0x30b261d, (0x1245 bytes) 2024-09-16T14:08:42.538 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.538 INFO:teuthology.orchestra.run.smithi079.stdout:4161 write 0x30a8dcf thru 0x30acfed (0x421f bytes) 2024-09-16T14:08:42.544 INFO:teuthology.orchestra.run.smithi079.stdout:4163 punch from 0x8c06a44 to 0x8c1353d, (0xcaf9 bytes) 2024-09-16T14:08:42.544 INFO:teuthology.orchestra.run.smithi079.stdout:4165 trunc from 0xa068288 to 0xa1f389a 2024-09-16T14:08:42.553 INFO:teuthology.orchestra.run.smithi079.stdout:4166 write 0x726e86e thru 0x726f37f (0xb12 bytes) 2024-09-16T14:08:42.558 INFO:teuthology.orchestra.run.smithi079.stdout:4167 writesame 0x959499f thru 0x95958c8 data_size 0x3(0xf2a bytes) 2024-09-16T14:08:42.563 INFO:teuthology.orchestra.run.smithi079.stdout:4168 read 0x9b30ffc thru 0x9b33c0b (0x2c10 bytes) 2024-09-16T14:08:42.563 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.563 INFO:teuthology.orchestra.run.smithi079.stdout:4170 read 0x350a657 thru 0x350bcf5 (0x169f bytes) 2024-09-16T14:08:42.563 INFO:teuthology.orchestra.run.smithi079.stdout:4171 write 0xc33be9 thru 0xc427f8 (0xec10 bytes) 2024-09-16T14:08:42.567 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:42.564+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:42.571 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.571 INFO:teuthology.orchestra.run.smithi079.stdout:4174 write 0x454b9a5 thru 0x454f54a (0x3ba6 bytes) 2024-09-16T14:08:42.577 INFO:teuthology.orchestra.run.smithi079.stdout:4176 read 0x2428f73 thru 0x2430135 (0x71c3 bytes) 2024-09-16T14:08:42.577 INFO:teuthology.orchestra.run.smithi079.stdout:4179 read 0x5966a3e thru 0x596bd5d (0x5320 bytes) 2024-09-16T14:08:42.577 INFO:teuthology.orchestra.run.smithi079.stdout:4180 read 0x33570c4 thru 0x33593ea (0x2327 bytes) 2024-09-16T14:08:42.577 INFO:teuthology.orchestra.run.smithi079.stdout:4182 writesame 0x45a528c thru 0x45aa99f data_size 0x2(0x5714 bytes) 2024-09-16T14:08:42.581 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:42.581 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:42.581 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:42.581 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:42.581 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:42.584 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:42.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:42.593 INFO:teuthology.orchestra.run.smithi079.stdout:4183 writesame 0x6519552 thru 0x6521daf data_size 0x5(0x885e bytes) 2024-09-16T14:08:42.603 INFO:teuthology.orchestra.run.smithi079.stdout:4186 read 0x776d83e thru 0x777a4c1 (0xcc84 bytes) 2024-09-16T14:08:42.603 INFO:teuthology.orchestra.run.smithi079.stdout:4187 read 0x200f507 thru 0x201015d (0xc57 bytes) 2024-09-16T14:08:42.605 INFO:teuthology.orchestra.run.smithi079.stdout:4188 trunc from 0xa1f389a to 0x8eaf243 2024-09-16T14:08:42.613 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:42.621 INFO:teuthology.orchestra.run.smithi079.stdout:4189 read 0x3df978e thru 0x3e06966 (0xd1d9 bytes) 2024-09-16T14:08:42.636 INFO:teuthology.orchestra.run.smithi079.stdout:4190 writesame 0xdb08f07 thru 0xdb10caa data_size 0x56(0x7da4 bytes) 2024-09-16T14:08:42.637 INFO:teuthology.orchestra.run.smithi079.stdout:4192 read 0x18c1673 thru 0x18c4243 (0x2bd1 bytes) 2024-09-16T14:08:42.637 INFO:teuthology.orchestra.run.smithi079.stdout:4193 read 0x53d3939 thru 0x53d3c05 (0x2cd bytes) 2024-09-16T14:08:42.637 INFO:teuthology.orchestra.run.smithi079.stdout:4195 punch from 0xb094983 to 0xb09c1a6, (0x7823 bytes) 2024-09-16T14:08:42.637 INFO:teuthology.orchestra.run.smithi079.stdout:4196 trunc from 0xdb10cab to 0xdf7c559 2024-09-16T14:08:42.645 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:42.645 INFO:teuthology.orchestra.run.smithi079.stdout:4197 read 0x5e7cab1 thru 0x5e7dd98 (0x12e8 bytes) 2024-09-16T14:08:42.645 INFO:teuthology.orchestra.run.smithi079.stdout:4200 punch from 0x41decd to 0x429e7c, (0xbfaf bytes) 2024-09-16T14:08:42.646 INFO:teuthology.orchestra.run.smithi079.stdout:4202 read 0xc334911 thru 0xc33a348 (0x5a38 bytes) 2024-09-16T14:08:42.646 INFO:teuthology.orchestra.run.smithi079.stdout:4203 writesame 0x2e729a7 thru 0x2e81536 data_size 0x10(0xeb90 bytes) 2024-09-16T14:08:42.663 INFO:teuthology.orchestra.run.smithi079.stdout:4204 write 0xdff95d6 thru 0xe00233a (0x8d65 bytes) 2024-09-16T14:08:42.668 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:42.664+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:42.669 INFO:teuthology.orchestra.run.smithi079.stdout:4207 read 0xca4211c thru 0xca43fe0 (0x1ec5 bytes) 2024-09-16T14:08:42.672 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:42.677 INFO:teuthology.orchestra.run.smithi079.stdout:4208 writesame 0xe09c5ca thru 0xe0a56ca data_size 0x1(0x9101 bytes) 2024-09-16T14:08:42.680 INFO:teuthology.orchestra.run.smithi079.stdout:4209 read 0x3794b7d thru 0x3799880 (0x4d04 bytes) 2024-09-16T14:08:42.680 INFO:teuthology.orchestra.run.smithi079.stdout:4210 write 0xa4d8f66 thru 0xa4e2566 (0x9601 bytes) 2024-09-16T14:08:42.686 INFO:teuthology.orchestra.run.smithi079.stdout:4213 writesame 0xc64890a thru 0xc64d31c data_size 0x3f(0x4a13 bytes) 2024-09-16T14:08:42.704 INFO:teuthology.orchestra.run.smithi079.stdout:4214 trunc from 0xe0a56cb to 0x4cfc441 2024-09-16T14:08:42.707 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:42.721 INFO:teuthology.orchestra.run.smithi079.stdout:4215 read 0x3935d75 thru 0x3936345 (0x5d1 bytes) 2024-09-16T14:08:42.721 INFO:teuthology.orchestra.run.smithi079.stdout:4217 write 0x3e7ee84 thru 0x3e8681c (0x7999 bytes) 2024-09-16T14:08:42.725 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:42.726 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:42.737 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:42.741 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:42.744 INFO:teuthology.orchestra.run.smithi079.stdout:4218 write 0x7ef7c09 thru 0x7f021a4 (0xa59c bytes) 2024-09-16T14:08:42.752 INFO:teuthology.orchestra.run.smithi079.stdout:4220 read 0x2eb15bb thru 0x2ebc0e1 (0xab27 bytes) 2024-09-16T14:08:42.752 INFO:teuthology.orchestra.run.smithi079.stdout:4221 read 0x76d9374 thru 0x76e715f (0xddec bytes) 2024-09-16T14:08:42.752 INFO:teuthology.orchestra.run.smithi079.stdout:4222 read 0x4265706 thru 0x42693ca (0x3cc5 bytes) 2024-09-16T14:08:42.752 INFO:teuthology.orchestra.run.smithi079.stdout:4224 punch from 0x4f048ea to 0x4f0f8f7, (0xb00d bytes) 2024-09-16T14:08:42.752 INFO:teuthology.orchestra.run.smithi079.stdout:4225 write 0x5d0a5b4 thru 0x5d18b2c (0xe579 bytes) 2024-09-16T14:08:42.768 INFO:teuthology.orchestra.run.smithi079.stdout:4226 write 0x93fd2e8 thru 0x940b324 (0xe03d bytes) 2024-09-16T14:08:42.769 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:42.764+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:42.775 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:42.779 INFO:teuthology.orchestra.run.smithi079.stdout:4228 read 0x9277c37 thru 0x928002a (0x83f4 bytes) 2024-09-16T14:08:42.779 INFO:teuthology.orchestra.run.smithi079.stdout:4229 writesame 0x570b3e9 thru 0x5715bb2 data_size 0x3(0xa7ca bytes) 2024-09-16T14:08:42.795 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:08:42.798 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:08:42.799 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:42.801 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:42.801 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:42.801 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:42.801 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:42.802 INFO:teuthology.orchestra.run.smithi079.stdout:4230 trunc from 0x940b325 to 0x1614a5b 2024-09-16T14:08:42.812 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-16T14:08:42.361657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1142755244,"num_objects":2760,"num_object_clones":606,"num_object_copies":5520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2760,"num_whiteouts":689,"num_read":208971,"num_read_kb":715504,"num_write":21764,"num_write_kb":351668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1272,"num_bytes_recovered":88263313,"num_keys_recovered":147,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16781,"ondisk_log_size":16781,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2180968,"kb_used_data":60100,"kb_used_omap":41,"kb_used_meta":2120534,"kb_avail":658421912,"statfs":{"total":676457349120,"available":674224037888,"internally_reserved":0,"allocated":61542400,"data_stored":533791598,"data_compressed":11370947,"data_compressed_allocated":40075264,"data_compressed_original":522233734,"omap_allocated":42088,"internal_metadata":2171427736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52379746,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":15,"num_read":5174,"num_read_kb":51885,"num_write":511,"num_write_kb":23449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":738,"num_bytes_recovered":30478468,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.953758"},"pg_stats":[{"pgid":"3.1f","version":"94'1609","reported_seq":5965,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.353015+0000","last_change":"2024-09-16T14:07:11.338271+0000","last_active":"2024-09-16T14:08:40.353015+0000","last_peered":"2024-09-16T14:08:40.353015+0000","last_clean":"2024-09-16T14:08:40.353015+0000","last_became_active":"2024-09-16T14:07:09.540900+0000","last_became_peered":"2024-09-16T14:07:09.540900+0000","last_unstale":"2024-09-16T14:08:40.353015+0000","last_undegraded":"2024-09-16T14:08:40.353015+0000","last_fullsized":"2024-09-16T14:08:40.353015+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1609,"log_dups_size":0,"ondisk_log_size":1609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:59.820596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233770000000001,"stat_sum":{"num_bytes":21475409,"num_objects":76,"num_object_clones":13,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":13,"num_read":13873,"num_read_kb":14589,"num_write":2761,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"94'571","reported_seq":2024,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.352855+0000","last_change":"2024-09-16T14:07:11.337654+0000","last_active":"2024-09-16T14:08:40.352855+0000","last_peered":"2024-09-16T14:08:40.352855+0000","last_clean":"2024-09-16T14:08:40.352855+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:08:40.352855+0000","last_undegraded":"2024-09-16T14:08:40.352855+0000","last_fullsized":"2024-09-16T14:08:40.352855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060911299999999999,"stat_sum":{"num_bytes":30011851,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":16,"num_read":3160,"num_read_kb":19744,"num_write":804,"num_write_kb":9017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"95'360","reported_seq":2789,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.376249+0000","last_change":"2024-09-16T14:08:28.327780+0000","last_active":"2024-09-16T14:08:40.376249+0000","last_peered":"2024-09-16T14:08:40.376249+0000","last_clean":"2024-09-16T14:08:40.376249+0000","last_became_active":"2024-09-16T14:08:28.215309+0000","last_became_peered":"2024-09-16T14:08:28.215309+0000","last_unstale":"2024-09-16T14:08:40.376249+0000","last_undegraded":"2024-09-16T14:08:40.376249+0000","last_fullsized":"2024-09-16T14:08:40.376249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:42.429361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072576699999999999,"stat_sum":{"num_bytes":34953713,"num_objects":93,"num_object_clones":20,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":19,"num_read":6059,"num_read_kb":20921,"num_write":534,"num_write_kb":12461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":5718086,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"94'699","reported_seq":13096,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.353408+0000","last_change":"2024-09-16T14:07:11.340416+0000","last_active":"2024-09-16T14:08:40.353408+0000","last_peered":"2024-09-16T14:08:40.353408+0000","last_clean":"2024-09-16T14:08:40.353408+0000","last_became_active":"2024-09-16T14:07:09.551657+0000","last_became_peered":"2024-09-16T14:07:09.551657+0000","last_unstale":"2024-09-16T14:08:40.353408+0000","last_undegraded":"2024-09-16T14:08:40.353408+0000","last_fullsized":"2024-09-16T14:08:40.353408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:49.635426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064309800000000004,"stat_sum":{"num_bytes":37997964,"num_objects":83,"num_object_clones":21,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":24,"num_read":14573,"num_read_kb":24305,"num_write":994,"num_write_kb":8334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"94'406","reported_seq":3112,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.355943+0000","last_change":"2024-09-16T14:07:22.350217+0000","last_active":"2024-09-16T14:08:40.355943+0000","last_peered":"2024-09-16T14:08:40.355943+0000","last_clean":"2024-09-16T14:08:40.355943+0000","last_became_active":"2024-09-16T14:07:22.349546+0000","last_became_peered":"2024-09-16T14:07:22.349546+0000","last_unstale":"2024-09-16T14:08:40.355943+0000","last_undegraded":"2024-09-16T14:08:40.355943+0000","last_fullsized":"2024-09-16T14:08:40.355943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:30.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021275310000000002,"stat_sum":{"num_bytes":15390451,"num_objects":82,"num_object_clones":12,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":19,"num_read":4953,"num_read_kb":11336,"num_write":463,"num_write_kb":5696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"96'416","reported_seq":4762,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.488049+0000","last_change":"2024-09-16T14:07:11.340816+0000","last_active":"2024-09-16T14:08:41.488049+0000","last_peered":"2024-09-16T14:08:41.488049+0000","last_clean":"2024-09-16T14:08:41.488049+0000","last_became_active":"2024-09-16T14:07:09.540237+0000","last_became_peered":"2024-09-16T14:07:09.540237+0000","last_unstale":"2024-09-16T14:08:41.488049+0000","last_undegraded":"2024-09-16T14:08:41.488049+0000","last_fullsized":"2024-09-16T14:08:41.488049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:38.147579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099540499999999999,"stat_sum":{"num_bytes":22667756,"num_objects":81,"num_object_clones":19,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":22,"num_read":5188,"num_read_kb":23788,"num_write":431,"num_write_kb":21793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"95'331","reported_seq":968,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.401994+0000","last_change":"2024-09-16T14:07:22.346713+0000","last_active":"2024-09-16T14:08:40.401994+0000","last_peered":"2024-09-16T14:08:40.401994+0000","last_clean":"2024-09-16T14:08:40.401994+0000","last_became_active":"2024-09-16T14:07:22.346031+0000","last_became_peered":"2024-09-16T14:07:22.346031+0000","last_unstale":"2024-09-16T14:08:40.401994+0000","last_undegraded":"2024-09-16T14:08:40.401994+0000","last_fullsized":"2024-09-16T14:08:40.401994+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:39.265638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003350142,"stat_sum":{"num_bytes":29690093,"num_objects":72,"num_object_clones":14,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":16,"num_read":963,"num_read_kb":20453,"num_write":378,"num_write_kb":16829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"96'405","reported_seq":5117,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.362819+0000","last_change":"2024-09-16T14:07:11.338172+0000","last_active":"2024-09-16T14:08:41.362819+0000","last_peered":"2024-09-16T14:08:41.362819+0000","last_clean":"2024-09-16T14:08:41.362819+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:08:41.362819+0000","last_undegraded":"2024-09-16T14:08:41.362819+0000","last_fullsized":"2024-09-16T14:08:41.362819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148865,"stat_sum":{"num_bytes":37726294,"num_objects":67,"num_object_clones":12,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":14,"num_read":6616,"num_read_kb":19690,"num_write":417,"num_write_kb":7376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"94'1126","reported_seq":4478,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.353191+0000","last_change":"2024-09-16T14:08:31.367987+0000","last_active":"2024-09-16T14:08:40.353191+0000","last_peered":"2024-09-16T14:08:40.353191+0000","last_clean":"2024-09-16T14:08:40.353191+0000","last_became_active":"2024-09-16T14:08:31.364152+0000","last_became_peered":"2024-09-16T14:08:31.364152+0000","last_unstale":"2024-09-16T14:08:40.353191+0000","last_undegraded":"2024-09-16T14:08:40.353191+0000","last_fullsized":"2024-09-16T14:08:40.353191+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:05.530637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215788,"stat_sum":{"num_bytes":30451218,"num_objects":84,"num_object_clones":21,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":24,"num_read":8098,"num_read_kb":32698,"num_write":1828,"num_write_kb":4763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1214676,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"96'426","reported_seq":4321,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.363498+0000","last_change":"2024-09-16T14:08:32.368069+0000","last_active":"2024-09-16T14:08:41.363498+0000","last_peered":"2024-09-16T14:08:41.363498+0000","last_clean":"2024-09-16T14:08:41.363498+0000","last_became_active":"2024-09-16T14:08:32.365885+0000","last_became_peered":"2024-09-16T14:08:32.365885+0000","last_unstale":"2024-09-16T14:08:41.363498+0000","last_undegraded":"2024-09-16T14:08:41.363498+0000","last_fullsized":"2024-09-16T14:08:41.363498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:53.426423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500454,"stat_sum":{"num_bytes":30391070,"num_objects":99,"num_object_clones":21,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":27,"num_read":5273,"num_read_kb":17950,"num_write":447,"num_write_kb":8457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":3589265,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"95'293","reported_seq":3891,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.402869+0000","last_change":"2024-09-16T14:07:11.343447+0000","last_active":"2024-09-16T14:08:40.402869+0000","last_peered":"2024-09-16T14:08:40.402869+0000","last_clean":"2024-09-16T14:08:40.402869+0000","last_became_active":"2024-09-16T14:07:09.549545+0000","last_became_peered":"2024-09-16T14:07:09.549545+0000","last_unstale":"2024-09-16T14:08:40.402869+0000","last_undegraded":"2024-09-16T14:08:40.402869+0000","last_fullsized":"2024-09-16T14:08:40.402869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:27.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012153120000000001,"stat_sum":{"num_bytes":33733809,"num_objects":91,"num_object_clones":28,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":29,"num_read":3645,"num_read_kb":23236,"num_write":314,"num_write_kb":6161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"94'400","reported_seq":2343,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.299766+0000","last_change":"2024-09-16T14:08:30.361920+0000","last_active":"2024-09-16T14:08:40.299766+0000","last_peered":"2024-09-16T14:08:40.299766+0000","last_clean":"2024-09-16T14:08:40.299766+0000","last_became_active":"2024-09-16T14:08:30.360514+0000","last_became_peered":"2024-09-16T14:08:30.360514+0000","last_unstale":"2024-09-16T14:08:40.299766+0000","last_undegraded":"2024-09-16T14:08:40.299766+0000","last_fullsized":"2024-09-16T14:08:40.299766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:15.505375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009984429999999999,"stat_sum":{"num_bytes":36911560,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":16,"num_read":3336,"num_read_kb":20888,"num_write":436,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":1548718,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T14:08:39.475561+0000","last_change":"2024-09-16T14:08:28.215419+0000","last_active":"2024-09-16T14:08:39.475561+0000","last_peered":"2024-09-16T14:08:39.475561+0000","last_clean":"2024-09-16T14:08:39.475561+0000","last_became_active":"2024-09-16T14:08:28.215112+0000","last_became_peered":"2024-09-16T14:08:28.215112+0000","last_unstale":"2024-09-16T14:08:39.475561+0000","last_undegraded":"2024-09-16T14:08:39.475561+0000","last_fullsized":"2024-09-16T14:08:39.475561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:19.841089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'535","reported_seq":3746,"reported_epoch":95,"state":"peering","last_fresh":"2024-09-16T14:08:40.355598+0000","last_change":"2024-09-16T14:08:40.356276+0000","last_active":"2024-09-16T14:08:40.355545+0000","last_peered":"2024-09-16T14:08:40.308852+0000","last_clean":"2024-09-16T14:08:36.920912+0000","last_became_active":"2024-09-16T14:08:39.439129+0000","last_became_peered":"2024-09-16T14:08:39.439129+0000","last_unstale":"2024-09-16T14:08:40.355598+0000","last_undegraded":"2024-09-16T14:08:40.355598+0000","last_fullsized":"2024-09-16T14:08:40.355598+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:16.514697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032490359999999999,"stat_sum":{"num_bytes":43378444,"num_objects":92,"num_object_clones":19,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":15,"num_read":4594,"num_read_kb":20738,"num_write":672,"num_write_kb":8160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":3405893,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.356511+0000","last_change":"2024-09-16T14:07:22.350069+0000","last_active":"2024-09-16T14:08:40.356511+0000","last_peered":"2024-09-16T14:08:40.356511+0000","last_clean":"2024-09-16T14:08:40.356511+0000","last_became_active":"2024-09-16T14:07:22.349757+0000","last_became_peered":"2024-09-16T14:07:22.349757+0000","last_unstale":"2024-09-16T14:08:40.356511+0000","last_undegraded":"2024-09-16T14:08:40.356511+0000","last_fullsized":"2024-09-16T14:08:40.356511+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:57.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"95'729","reported_seq":6359,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.402290+0000","last_change":"2024-09-16T14:07:22.351951+0000","last_active":"2024-09-16T14:08:40.402290+0000","last_peered":"2024-09-16T14:08:40.402290+0000","last_clean":"2024-09-16T14:08:40.402290+0000","last_became_active":"2024-09-16T14:07:22.351394+0000","last_became_peered":"2024-09-16T14:07:22.351394+0000","last_unstale":"2024-09-16T14:08:40.402290+0000","last_undegraded":"2024-09-16T14:08:40.402290+0000","last_fullsized":"2024-09-16T14:08:40.402290+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:55.945287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027484229999999998,"stat_sum":{"num_bytes":76166909,"num_objects":91,"num_object_clones":20,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":25,"num_read":9056,"num_read_kb":41302,"num_write":1080,"num_write_kb":28674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T14:08:39.475076+0000","last_change":"2024-09-16T14:08:28.213074+0000","last_active":"2024-09-16T14:08:39.475076+0000","last_peered":"2024-09-16T14:08:39.475076+0000","last_clean":"2024-09-16T14:08:39.475076+0000","last_became_active":"2024-09-16T14:08:28.212640+0000","last_became_peered":"2024-09-16T14:08:28.212640+0000","last_unstale":"2024-09-16T14:08:39.475076+0000","last_undegraded":"2024-09-16T14:08:39.475076+0000","last_fullsized":"2024-09-16T14:08:39.475076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:09.532319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'382","reported_seq":3074,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.362663+0000","last_change":"2024-09-16T14:07:11.547417+0000","last_active":"2024-09-16T14:08:41.362663+0000","last_peered":"2024-09-16T14:08:41.362663+0000","last_clean":"2024-09-16T14:08:41.362663+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:08:41.362663+0000","last_undegraded":"2024-09-16T14:08:41.362663+0000","last_fullsized":"2024-09-16T14:08:41.362663+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097608199999999997,"stat_sum":{"num_bytes":16269933,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":25,"num_read":3943,"num_read_kb":12340,"num_write":414,"num_write_kb":5398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":89,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.352859+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:08:40.352859+0000","last_peered":"2024-09-16T14:08:40.352859+0000","last_clean":"2024-09-16T14:08:40.352859+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:08:40.352859+0000","last_undegraded":"2024-09-16T14:08:40.352859+0000","last_fullsized":"2024-09-16T14:08:40.352859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"95'520","reported_seq":8476,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.402745+0000","last_change":"2024-09-16T14:07:11.361331+0000","last_active":"2024-09-16T14:08:40.402745+0000","last_peered":"2024-09-16T14:08:40.402745+0000","last_clean":"2024-09-16T14:08:40.402745+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:08:40.402745+0000","last_undegraded":"2024-09-16T14:08:40.402745+0000","last_fullsized":"2024-09-16T14:08:40.402745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062220600000000004,"stat_sum":{"num_bytes":25326929,"num_objects":89,"num_object_clones":19,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":22,"num_read":9248,"num_read_kb":10735,"num_write":661,"num_write_kb":4475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"96'640","reported_seq":3379,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.523929+0000","last_change":"2024-09-16T14:07:11.342816+0000","last_active":"2024-09-16T14:08:41.523929+0000","last_peered":"2024-09-16T14:08:41.523929+0000","last_clean":"2024-09-16T14:08:41.523929+0000","last_became_active":"2024-09-16T14:07:09.549842+0000","last_became_peered":"2024-09-16T14:07:09.549842+0000","last_unstale":"2024-09-16T14:08:41.523929+0000","last_undegraded":"2024-09-16T14:08:41.523929+0000","last_fullsized":"2024-09-16T14:08:41.523929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:47.724999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014467779999999999,"stat_sum":{"num_bytes":36152803,"num_objects":91,"num_object_clones":18,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":26,"num_read":6632,"num_read_kb":14410,"num_write":754,"num_write_kb":8203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T14:08:39.474876+0000","last_change":"2024-09-16T14:08:28.217645+0000","last_active":"2024-09-16T14:08:39.474876+0000","last_peered":"2024-09-16T14:08:39.474876+0000","last_clean":"2024-09-16T14:08:39.474876+0000","last_became_active":"2024-09-16T14:08:28.217418+0000","last_became_peered":"2024-09-16T14:08:28.217418+0000","last_unstale":"2024-09-16T14:08:39.474876+0000","last_undegraded":"2024-09-16T14:08:39.474876+0000","last_fullsized":"2024-09-16T14:08:39.474876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:25.918964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"94'981","reported_seq":4404,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.367878+0000","last_change":"2024-09-16T14:08:41.367878+0000","last_active":"2024-09-16T14:08:41.367878+0000","last_peered":"2024-09-16T14:08:41.367878+0000","last_clean":"2024-09-16T14:08:41.367878+0000","last_became_active":"2024-09-16T14:08:41.365767+0000","last_became_peered":"2024-09-16T14:08:41.365767+0000","last_unstale":"2024-09-16T14:08:41.367878+0000","last_undegraded":"2024-09-16T14:08:41.367878+0000","last_fullsized":"2024-09-16T14:08:41.367878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:19:25.019511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076943300000000003,"stat_sum":{"num_bytes":50574402,"num_objects":108,"num_object_clones":26,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":31,"num_read":6594,"num_read_kb":21674,"num_write":1483,"num_write_kb":9428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":7223267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.352807+0000","last_change":"2024-09-16T14:08:38.421227+0000","last_active":"2024-09-16T14:08:40.352807+0000","last_peered":"2024-09-16T14:08:40.352807+0000","last_clean":"2024-09-16T14:08:40.352807+0000","last_became_active":"2024-09-16T14:08:38.420867+0000","last_became_peered":"2024-09-16T14:08:38.420867+0000","last_unstale":"2024-09-16T14:08:40.352807+0000","last_undegraded":"2024-09-16T14:08:40.352807+0000","last_fullsized":"2024-09-16T14:08:40.352807+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:52.563336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"94'579","reported_seq":6125,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.357706+0000","last_change":"2024-09-16T14:07:11.547270+0000","last_active":"2024-09-16T14:08:40.357706+0000","last_peered":"2024-09-16T14:08:40.357706+0000","last_clean":"2024-09-16T14:08:40.357706+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:08:40.357706+0000","last_undegraded":"2024-09-16T14:08:40.357706+0000","last_fullsized":"2024-09-16T14:08:40.357706+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085373199999999995,"stat_sum":{"num_bytes":34643670,"num_objects":84,"num_object_clones":17,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":22,"num_read":7517,"num_read_kb":19408,"num_write":774,"num_write_kb":12765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T14:08:39.474549+0000","last_change":"2024-09-16T14:08:28.214757+0000","last_active":"2024-09-16T14:08:39.474549+0000","last_peered":"2024-09-16T14:08:39.474549+0000","last_clean":"2024-09-16T14:08:39.474549+0000","last_became_active":"2024-09-16T14:08:28.213488+0000","last_became_peered":"2024-09-16T14:08:28.213488+0000","last_unstale":"2024-09-16T14:08:39.474549+0000","last_undegraded":"2024-09-16T14:08:39.474549+0000","last_fullsized":"2024-09-16T14:08:39.474549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:11.596666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"95'350","reported_seq":10523,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.403777+0000","last_change":"2024-09-16T14:07:22.350954+0000","last_active":"2024-09-16T14:08:40.403777+0000","last_peered":"2024-09-16T14:08:40.403777+0000","last_clean":"2024-09-16T14:08:40.403777+0000","last_became_active":"2024-09-16T14:07:22.350024+0000","last_became_peered":"2024-09-16T14:07:22.350024+0000","last_unstale":"2024-09-16T14:08:40.403777+0000","last_undegraded":"2024-09-16T14:08:40.403777+0000","last_fullsized":"2024-09-16T14:08:40.403777+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:07.634077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002704522,"stat_sum":{"num_bytes":39156526,"num_objects":87,"num_object_clones":25,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":20,"num_read":10259,"num_read_kb":36256,"num_write":306,"num_write_kb":7978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T14:08:39.474617+0000","last_change":"2024-09-16T14:08:28.215537+0000","last_active":"2024-09-16T14:08:39.474617+0000","last_peered":"2024-09-16T14:08:39.474617+0000","last_clean":"2024-09-16T14:08:39.474617+0000","last_became_active":"2024-09-16T14:08:28.214888+0000","last_became_peered":"2024-09-16T14:08:28.214888+0000","last_unstale":"2024-09-16T14:08:39.474617+0000","last_undegraded":"2024-09-16T14:08:39.474617+0000","last_fullsized":"2024-09-16T14:08:39.474617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:23.766012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":177,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.354602+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:08:40.354602+0000","last_peered":"2024-09-16T14:08:40.354602+0000","last_clean":"2024-09-16T14:08:40.354602+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:08:40.354602+0000","last_undegraded":"2024-09-16T14:08:40.354602+0000","last_fullsized":"2024-09-16T14:08:40.354602+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"94'430","reported_seq":2755,"reported_epoch":96,"state":"creating+peering","last_fresh":"2024-09-16T14:08:41.364090+0000","last_change":"2024-09-16T14:08:41.363796+0000","last_active":"2024-09-16T14:07:08.513326+0000","last_peered":"2024-09-16T14:07:08.513326+0000","last_clean":"2024-09-16T14:07:08.513326+0000","last_became_active":"2024-09-16T14:08:39.453653+0000","last_became_peered":"2024-09-16T14:08:39.453653+0000","last_unstale":"2024-09-16T14:08:41.364090+0000","last_undegraded":"2024-09-16T14:08:41.364090+0000","last_fullsized":"2024-09-16T14:08:41.364090+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:14.078847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022177080000000001,"stat_sum":{"num_bytes":19615050,"num_objects":99,"num_object_clones":19,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":28,"num_read":2709,"num_read_kb":10390,"num_write":506,"num_write_kb":5293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":1304166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"95'391","reported_seq":5422,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.375815+0000","last_change":"2024-09-16T14:08:30.360750+0000","last_active":"2024-09-16T14:08:40.375815+0000","last_peered":"2024-09-16T14:08:40.375815+0000","last_clean":"2024-09-16T14:08:40.375815+0000","last_became_active":"2024-09-16T14:08:30.357783+0000","last_became_peered":"2024-09-16T14:08:30.357783+0000","last_unstale":"2024-09-16T14:08:40.375815+0000","last_undegraded":"2024-09-16T14:08:40.375815+0000","last_fullsized":"2024-09-16T14:08:40.375815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:45.786220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339700000000001,"stat_sum":{"num_bytes":20037421,"num_objects":90,"num_object_clones":18,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":23,"num_read":6206,"num_read_kb":14649,"num_write":404,"num_write_kb":8398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2583563,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"94'527","reported_seq":5513,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.302983+0000","last_change":"2024-09-16T14:08:30.359428+0000","last_active":"2024-09-16T14:08:40.302983+0000","last_peered":"2024-09-16T14:08:40.302983+0000","last_clean":"2024-09-16T14:08:40.302983+0000","last_became_active":"2024-09-16T14:08:30.356626+0000","last_became_peered":"2024-09-16T14:08:30.356626+0000","last_unstale":"2024-09-16T14:08:40.302983+0000","last_undegraded":"2024-09-16T14:08:40.302983+0000","last_fullsized":"2024-09-16T14:08:40.302983+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:50.638542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065751299999999998,"stat_sum":{"num_bytes":71524472,"num_objects":104,"num_object_clones":25,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":28,"num_read":6515,"num_read_kb":34193,"num_write":534,"num_write_kb":17571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":14526461,"num_keys_recovered":40,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"96'473","reported_seq":3599,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.370847+0000","last_change":"2024-09-16T14:08:41.367794+0000","last_active":"2024-09-16T14:08:41.370847+0000","last_peered":"2024-09-16T14:08:41.370847+0000","last_clean":"2024-09-16T14:08:41.370847+0000","last_became_active":"2024-09-16T14:08:41.366106+0000","last_became_peered":"2024-09-16T14:08:41.366106+0000","last_unstale":"2024-09-16T14:08:41.370847+0000","last_undegraded":"2024-09-16T14:08:41.370847+0000","last_fullsized":"2024-09-16T14:08:41.370847+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:15.221267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056925200000000002,"stat_sum":{"num_bytes":27570071,"num_objects":77,"num_object_clones":20,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":22,"num_read":5162,"num_read_kb":40479,"num_write":597,"num_write_kb":21439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":9737480,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"96'371","reported_seq":6254,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.366192+0000","last_change":"2024-09-16T14:08:41.365299+0000","last_active":"2024-09-16T14:08:41.366192+0000","last_peered":"2024-09-16T14:08:41.366192+0000","last_clean":"2024-09-16T14:08:41.366192+0000","last_became_active":"2024-09-16T14:08:41.363613+0000","last_became_peered":"2024-09-16T14:08:41.363613+0000","last_unstale":"2024-09-16T14:08:41.366192+0000","last_undegraded":"2024-09-16T14:08:41.366192+0000","last_fullsized":"2024-09-16T14:08:41.366192+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:22.795636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017881959999999999,"stat_sum":{"num_bytes":34599134,"num_objects":101,"num_object_clones":22,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":31,"num_read":6145,"num_read_kb":26557,"num_write":334,"num_write_kb":12030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":8807662,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"95'253","reported_seq":1362,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.376626+0000","last_change":"2024-09-16T14:08:28.444504+0000","last_active":"2024-09-16T14:08:40.376626+0000","last_peered":"2024-09-16T14:08:40.376626+0000","last_clean":"2024-09-16T14:08:40.376626+0000","last_became_active":"2024-09-16T14:08:28.220167+0000","last_became_peered":"2024-09-16T14:08:28.220167+0000","last_unstale":"2024-09-16T14:08:40.376626+0000","last_undegraded":"2024-09-16T14:08:40.376626+0000","last_fullsized":"2024-09-16T14:08:40.376626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:40.805140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074836900000000003,"stat_sum":{"num_bytes":30990066,"num_objects":78,"num_object_clones":19,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":18,"num_read":7239,"num_read_kb":17660,"num_write":215,"num_write_kb":4830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":5458762,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"91'236","reported_seq":2669,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.358236+0000","last_change":"2024-09-16T14:07:11.362189+0000","last_active":"2024-09-16T14:08:40.358236+0000","last_peered":"2024-09-16T14:08:40.358236+0000","last_clean":"2024-09-16T14:08:40.358236+0000","last_became_active":"2024-09-16T14:07:09.548973+0000","last_became_peered":"2024-09-16T14:07:09.548973+0000","last_unstale":"2024-09-16T14:08:40.358236+0000","last_undegraded":"2024-09-16T14:08:40.358236+0000","last_fullsized":"2024-09-16T14:08:40.358236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:17.345008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013505660000000001,"stat_sum":{"num_bytes":25425605,"num_objects":73,"num_object_clones":17,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":19,"num_read":3155,"num_read_kb":10525,"num_write":206,"num_write_kb":4884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"91'712","reported_seq":5303,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.356616+0000","last_change":"2024-09-16T14:07:11.342735+0000","last_active":"2024-09-16T14:08:40.356616+0000","last_peered":"2024-09-16T14:08:40.356616+0000","last_clean":"2024-09-16T14:08:40.356616+0000","last_became_active":"2024-09-16T14:07:09.559054+0000","last_became_peered":"2024-09-16T14:07:09.559054+0000","last_unstale":"2024-09-16T14:08:40.356616+0000","last_undegraded":"2024-09-16T14:08:40.356616+0000","last_fullsized":"2024-09-16T14:08:40.356616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.926445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372222,"stat_sum":{"num_bytes":42585558,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":18,"num_read":8691,"num_read_kb":17654,"num_write":973,"num_write_kb":9134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"96'353","reported_seq":10860,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.511664+0000","last_change":"2024-09-16T14:07:11.340733+0000","last_active":"2024-09-16T14:08:41.511664+0000","last_peered":"2024-09-16T14:08:41.511664+0000","last_clean":"2024-09-16T14:08:41.511664+0000","last_became_active":"2024-09-16T14:07:09.540341+0000","last_became_peered":"2024-09-16T14:07:09.540341+0000","last_unstale":"2024-09-16T14:08:41.511664+0000","last_undegraded":"2024-09-16T14:08:41.511664+0000","last_fullsized":"2024-09-16T14:08:41.511664+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:18.953079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012161470000000001,"stat_sum":{"num_bytes":47334823,"num_objects":87,"num_object_clones":21,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":20,"num_read":11302,"num_read_kb":20827,"num_write":547,"num_write_kb":13850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"94'431","reported_seq":4094,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.300668+0000","last_change":"2024-09-16T14:08:32.368616+0000","last_active":"2024-09-16T14:08:40.300668+0000","last_peered":"2024-09-16T14:08:40.300668+0000","last_clean":"2024-09-16T14:08:40.300668+0000","last_became_active":"2024-09-16T14:08:32.366601+0000","last_became_peered":"2024-09-16T14:08:32.366601+0000","last_unstale":"2024-09-16T14:08:40.300668+0000","last_undegraded":"2024-09-16T14:08:40.300668+0000","last_fullsized":"2024-09-16T14:08:40.300668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:01.897916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037127800000000001,"stat_sum":{"num_bytes":19414460,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":14,"num_read":4552,"num_read_kb":11749,"num_write":533,"num_write_kb":8097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":635545,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"96'461","reported_seq":5091,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-16T14:08:41.362515+0000","last_change":"2024-09-16T14:08:30.363925+0000","last_active":"2024-09-16T14:08:41.362515+0000","last_peered":"2024-09-16T14:08:41.362515+0000","last_clean":"2024-09-16T14:08:41.362515+0000","last_became_active":"2024-09-16T14:08:30.360870+0000","last_became_peered":"2024-09-16T14:08:30.360870+0000","last_unstale":"2024-09-16T14:08:41.362515+0000","last_undegraded":"2024-09-16T14:08:41.362515+0000","last_fullsized":"2024-09-16T14:08:41.362515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:04.684260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012662420000000001,"stat_sum":{"num_bytes":59739758,"num_objects":90,"num_object_clones":17,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":22,"num_read":6822,"num_read_kb":54338,"num_write":501,"num_write_kb":28092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":11455145,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"95'310","reported_seq":6286,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-16T14:08:40.358839+0000","last_change":"2024-09-16T14:08:30.359625+0000","last_active":"2024-09-16T14:08:40.358839+0000","last_peered":"2024-09-16T14:08:40.358839+0000","last_clean":"2024-09-16T14:08:40.358839+0000","last_became_active":"2024-09-16T14:08:30.357685+0000","last_became_peered":"2024-09-16T14:08:30.357685+0000","last_unstale":"2024-09-16T14:08:40.358839+0000","last_undegraded":"2024-09-16T14:08:40.358839+0000","last_fullsized":"2024-09-16T14:08:40.358839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:26.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066482800000000001,"stat_sum":{"num_bytes":60257635,"num_objects":86,"num_object_clones":22,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":21,"num_read":6827,"num_read_kb":29967,"num_write":360,"num_write_kb":17263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":11054624,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1142164857,"num_objects":2756,"num_object_clones":606,"num_object_copies":5512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2756,"num_whiteouts":689,"num_read":208905,"num_read_kb":715449,"num_write":21661,"num_write_kb":350038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1272,"num_bytes_recovered":88263313,"num_keys_recovered":147,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":54075392,"data_stored":528571425,"data_compressed":11540776,"data_compressed_allocated":40165376,"data_compressed_original":520452318,"omap_allocated":8948,"internal_metadata":0},"log_size":16706,"ondisk_log_size":16706,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509466,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305784,"kb_used_data":8420,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066056,"statfs":{"total":96636764160,"available":96323641344,"internally_reserved":0,"allocated":8622080,"data_stored":89652738,"data_compressed":1675715,"data_compressed_allocated":5931008,"data_compressed_original":88268150,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78300000000000003}]},{"osd":1,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.52500000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.749},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.93600000000000005}]},{"osd":5,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.96599999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.0389999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312228,"kb_used_data":6488,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94059612,"statfs":{"total":96636764160,"available":96317042688,"internally_reserved":0,"allocated":6643712,"data_stored":53398216,"data_compressed":1079877,"data_compressed_allocated":3809280,"data_compressed_original":51855227,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.57799999999999996}]},{"osd":2,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.45200000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.315}]},{"osd":7,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.55200000000000005}]}]},{"osd":3,"up_from":83,"seq":356482285572,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314704,"kb_used_data":11404,"kb_used_omap":31,"kb_used_meta":303200,"kb_avail":94057136,"statfs":{"total":96636764160,"available":96314507264,"internally_reserved":0,"allocated":11677696,"data_stored":96219813,"data_compressed":2248792,"data_compressed_allocated":8118272,"data_compressed_original":94258087,"omap_allocated":31864,"internal_metadata":310477704},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309812,"kb_used_data":8300,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94062028,"statfs":{"total":96636764160,"available":96319516672,"internally_reserved":0,"allocated":8499200,"data_stored":70154563,"data_compressed":1567745,"data_compressed_allocated":5095424,"data_compressed_original":68294469,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":1.008}]},{"osd":5,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.95599999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.89300000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.98199999999999998}]}]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310344,"kb_used_data":8828,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94061496,"statfs":{"total":96636764160,"available":96318971904,"internally_reserved":0,"allocated":9039872,"data_stored":71155028,"data_compressed":1492954,"data_compressed_allocated":5926912,"data_compressed_original":69524829,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":1.0029999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.753},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.94499999999999995}]},{"osd":3,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.077},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.87}]},{"osd":4,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.153},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.84399999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.113}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.0449999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.196}]}]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306532,"kb_used_data":9096,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065308,"statfs":{"total":96636764160,"available":96322875392,"internally_reserved":0,"allocated":9314304,"data_stored":81828244,"data_compressed":1896951,"data_compressed_allocated":6115328,"data_compressed_original":80059504,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64300000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.82299999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.96499999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.0900000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.139}]}]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321564,"kb_used_data":7564,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94050276,"statfs":{"total":96636764160,"available":96307482624,"internally_reserved":0,"allocated":7745536,"data_stored":71382996,"data_compressed":1408913,"data_compressed_allocated":5079040,"data_compressed_original":69973468,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.46500000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.501}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.90200000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.92300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8232960,"data_stored":81344942,"data_compressed":1909797,"data_compressed_allocated":6135808,"data_compressed_original":80076813,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7880704,"data_stored":69044372,"data_compressed":1545807,"data_compressed_allocated":5881856,"data_compressed_original":67939096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7368704,"data_stored":69633538,"data_compressed":1567745,"data_compressed_allocated":5095424,"data_compressed_original":68294469,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":95704644,"data_compressed":2248792,"data_compressed_allocated":8118272,"data_compressed_original":94258087,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6463488,"data_stored":69211404,"data_compressed":1387286,"data_compressed_allocated":4935680,"data_compressed_original":68330972,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5517312,"data_stored":52348947,"data_compressed":1105298,"data_compressed_allocated":3809280,"data_compressed_original":51330939,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7565312,"data_stored":89226604,"data_compressed":1711610,"data_compressed_allocated":5980160,"data_compressed_original":88345974,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":2056974,"data_compressed":64441,"data_compressed_allocated":208896,"data_compressed_original":1875968,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:08:42.838 INFO:teuthology.orchestra.run.smithi079.stdout:4231 write 0x5933f92 thru 0x5935cd0 (0x1d3f bytes) 2024-09-16T14:08:42.844 INFO:teuthology.orchestra.run.smithi079.stdout:4233 read 0x2d5ef6f thru 0x2d68a59 (0x9aeb bytes) 2024-09-16T14:08:42.845 INFO:teuthology.orchestra.run.smithi079.stdout:4234 read 0x4228f16 thru 0x4237f2e (0xf019 bytes) 2024-09-16T14:08:42.845 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.845 INFO:teuthology.orchestra.run.smithi079.stdout:4237 read 0x4dd141e thru 0x4ddeca6 (0xd889 bytes) 2024-09-16T14:08:42.845 INFO:teuthology.orchestra.run.smithi079.stdout:4238 punch from 0x3b4d669 to 0x3b52a52, (0x53e9 bytes) 2024-09-16T14:08:42.850 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:42.850 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:42.851 INFO:teuthology.orchestra.run.smithi079.stdout:4239 trunc from 0x5935cd1 to 0x214b2d3 2024-09-16T14:08:42.860 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:42.869 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:42.862+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:42.870 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.888 INFO:teuthology.orchestra.run.smithi079.stdout:4242 write 0x59ff70a thru 0x5a0a5b3 (0xaeaa bytes) 2024-09-16T14:08:42.896 INFO:teuthology.orchestra.run.smithi079.stdout:4244 writesame 0x42ffca3 thru 0x430918e data_size 0x36(0x94ec bytes) 2024-09-16T14:08:42.916 INFO:teuthology.orchestra.run.smithi079.stdout:4245 write 0x5e004ae thru 0x5e04eff (0x4a52 bytes) 2024-09-16T14:08:42.921 INFO:teuthology.orchestra.run.smithi079.stdout:4246 write 0x2bfcd63 thru 0x2bff117 (0x23b5 bytes) 2024-09-16T14:08:42.927 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:42.927 INFO:teuthology.orchestra.run.smithi079.stdout:4250 write 0x2a4c923 thru 0x2a54c26 (0x8304 bytes) 2024-09-16T14:08:42.934 INFO:teuthology.orchestra.run.smithi079.stdout:4252 read 0x4852d5e thru 0x485c6c4 (0x9967 bytes) 2024-09-16T14:08:42.942 INFO:teuthology.orchestra.run.smithi079.stdout:4253 trunc from 0x5e04f00 to 0x278bb3d 2024-09-16T14:08:42.958 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:42.958 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:42.967 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:42.970 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:42.968+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:42.996 INFO:teuthology.orchestra.run.smithi079.stdout:4254 writesame 0xeb2ad32 thru 0xeb2f729 data_size 0x18(0x49f8 bytes) 2024-09-16T14:08:42.997 INFO:teuthology.orchestra.run.smithi079.stdout:4255 read 0xcf71a2 thru 0xcff937 (0x8796 bytes) 2024-09-16T14:08:42.997 INFO:teuthology.orchestra.run.smithi079.stdout:4256 write 0x8ec5790 thru 0x8ec9a22 (0x4293 bytes) 2024-09-16T14:08:43.002 INFO:teuthology.orchestra.run.smithi079.stdout:4257 writesame 0x1d9ff8c thru 0x1dacc24 data_size 0x4f(0xcc99 bytes) 2024-09-16T14:08:43.011 INFO:teuthology.orchestra.run.smithi079.stdout:4259 writesame 0x2655147 thru 0x2660bb4 data_size 0xe(0xba6e bytes) 2024-09-16T14:08:43.015 INFO:teuthology.orchestra.run.smithi079.stdout:4260 punch from 0x4864e5b to 0x486bbd3, (0x6d78 bytes) 2024-09-16T14:08:43.015 INFO:teuthology.orchestra.run.smithi079.stdout:4261 write 0xbc85b28 thru 0xbc94e72 (0xf34b bytes) 2024-09-16T14:08:43.024 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.024 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.024 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.024 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.024 INFO:teuthology.orchestra.run.smithi079.stdout:4268 write 0x184e354 thru 0x185d680 (0xf32d bytes) 2024-09-16T14:08:43.055 INFO:teuthology.orchestra.run.smithi079.stdout:4271 trunc from 0xeb2f72a to 0xa9525b 2024-09-16T14:08:43.067 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.067 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.071 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:43.068+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:43.077 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:43.104 INFO:teuthology.orchestra.run.smithi079.stdout:4272 write 0xa5be620 thru 0xa5be8d1 (0x2b2 bytes) 2024-09-16T14:08:43.109 INFO:teuthology.orchestra.run.smithi079.stdout:4275 write 0x8377392 thru 0x837923c (0x1eab bytes) 2024-09-16T14:08:43.127 INFO:teuthology.orchestra.run.smithi079.stdout:4277 write 0xab30fd7 thru 0xab3ecaa (0xdcd4 bytes) 2024-09-16T14:08:43.134 INFO:teuthology.orchestra.run.smithi079.stdout:4278 read 0x3802290 thru 0x380f958 (0xd6c9 bytes) 2024-09-16T14:08:43.134 INFO:teuthology.orchestra.run.smithi079.stdout:4280 read 0x5fd9616 thru 0x5fdb2f9 (0x1ce4 bytes) 2024-09-16T14:08:43.134 INFO:teuthology.orchestra.run.smithi079.stdout:4281 read 0x4cd7de thru 0x4d4900 (0x7123 bytes) 2024-09-16T14:08:43.135 INFO:teuthology.orchestra.run.smithi079.stdout:4282 write 0x312dd75 thru 0x312eb88 (0xe14 bytes) 2024-09-16T14:08:43.141 INFO:teuthology.orchestra.run.smithi079.stdout:4283 write 0x7ab075b thru 0x7aba1b6 (0x9a5c bytes) 2024-09-16T14:08:43.148 INFO:teuthology.orchestra.run.smithi079.stdout:4284 punch from 0x1ec3d05 to 0x1ecb4b6, (0x77b1 bytes) 2024-09-16T14:08:43.149 INFO:teuthology.orchestra.run.smithi079.stdout:4285 write 0x3ecbcd1 thru 0x3edaa3a (0xed6a bytes) 2024-09-16T14:08:43.172 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.173 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.173 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:43.168+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:43.174 INFO:teuthology.orchestra.run.smithi079.stdout:4286 trunc from 0xab3ecab to 0x212d627 2024-09-16T14:08:43.182 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:43.205 INFO:teuthology.orchestra.run.smithi079.stdout:4287 trunc from 0x212d627 to 0x89e5fc8 2024-09-16T14:08:43.216 INFO:teuthology.orchestra.run.smithi079.stdout:4288 read 0x6b3ad83 thru 0x6b47cfb (0xcf79 bytes) 2024-09-16T14:08:43.216 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.234 INFO:teuthology.orchestra.run.smithi079.stdout:4290 writesame 0xc628f12 thru 0xc62d193 data_size 0x1(0x4282 bytes) 2024-09-16T14:08:43.234 INFO:teuthology.orchestra.run.smithi079.stdout:4291 read 0xa3f3a87 thru 0xa40035f (0xc8d9 bytes) 2024-09-16T14:08:43.234 INFO:teuthology.orchestra.run.smithi079.stdout:4295 punch from 0x6a44b81 to 0x6a49c80, (0x50ff bytes) 2024-09-16T14:08:43.249 INFO:teuthology.orchestra.run.smithi079.stdout:4296 trunc from 0xc62d194 to 0x4016037 2024-09-16T14:08:43.260 INFO:teuthology.orchestra.run.smithi079.stdout:4297 punch from 0x1836786 to 0x183d66d, (0x6ee7 bytes) 2024-09-16T14:08:43.273 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:43.268+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:43.276 INFO:teuthology.orchestra.run.smithi079.stdout:4298 writesame 0x6a7ce98 thru 0x6a8125d data_size 0x19(0x43c6 bytes) 2024-09-16T14:08:43.278 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.279 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.288 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:43.301 INFO:teuthology.orchestra.run.smithi079.stdout:4299 write 0xd5715eb thru 0xd57edbf (0xd7d5 bytes) 2024-09-16T14:08:43.323 INFO:teuthology.orchestra.run.smithi079.stdout:4300 trunc from 0xd57edc0 to 0x46d58e9 2024-09-16T14:08:43.342 INFO:teuthology.orchestra.run.smithi079.stdout:4302 read 0x158af79 thru 0x159a308 (0xf390 bytes) 2024-09-16T14:08:43.342 INFO:teuthology.orchestra.run.smithi079.stdout:4303 punch from 0x1c167f7 to 0x1c21083, (0xa88c bytes) 2024-09-16T14:08:43.350 INFO:teuthology.orchestra.run.smithi079.stdout:4305 trunc from 0x46d58e9 to 0x8fd5bb1 2024-09-16T14:08:43.362 INFO:teuthology.orchestra.run.smithi079.stdout:4306 read 0x8760a4e thru 0x8770597 (0xfb4a bytes) 2024-09-16T14:08:43.362 INFO:teuthology.orchestra.run.smithi079.stdout:4307 writesame 0x610ab21 thru 0x611869e data_size 0xf(0xdb7e bytes) 2024-09-16T14:08:43.373 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:43.366+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:43.374 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:08:43.376 INFO:teuthology.orchestra.run.smithi079.stdout:4308 read 0x35ef6e5 thru 0x35f03d7 (0xcf3 bytes) 2024-09-16T14:08:43.376 INFO:teuthology.orchestra.run.smithi079.stdout:4309 read 0x554bb1e thru 0x5553c17 (0x80fa bytes) 2024-09-16T14:08:43.376 INFO:teuthology.orchestra.run.smithi079.stdout:4310 punch from 0x15644e7 to 0x15679b4, (0x34cd bytes) 2024-09-16T14:08:43.376 INFO:teuthology.orchestra.run.smithi079.stdout:4311 write 0x33365c1 thru 0x3345969 (0xf3a9 bytes) 2024-09-16T14:08:43.384 INFO:teuthology.orchestra.run.smithi079.stdout:4312 writesame 0x1a2bb3a thru 0x1a362ed data_size 0x2(0xa7b4 bytes) 2024-09-16T14:08:43.389 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.389 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:43.402 INFO:teuthology.orchestra.run.smithi079.stdout:4314 read 0x94e43a thru 0x956a96 (0x865d bytes) 2024-09-16T14:08:43.402 INFO:teuthology.orchestra.run.smithi079.stdout:4315 read 0x75615ac thru 0x75645c4 (0x3019 bytes) 2024-09-16T14:08:43.402 INFO:teuthology.orchestra.run.smithi079.stdout:4316 write 0x3768b80 thru 0x376ea3d (0x5ebe bytes) 2024-09-16T14:08:43.407 INFO:teuthology.orchestra.run.smithi079.stdout:4317 read 0x38b82f5 thru 0x38bd51e (0x522a bytes) 2024-09-16T14:08:43.407 INFO:teuthology.orchestra.run.smithi079.stdout:4318 write 0x630d88b thru 0x631b98b (0xe101 bytes) 2024-09-16T14:08:43.416 INFO:teuthology.orchestra.run.smithi079.stdout:4322 punch from 0x1fa366c to 0x1faa730, (0x70c4 bytes) 2024-09-16T14:08:43.424 INFO:teuthology.orchestra.run.smithi079.stdout:4323 trunc from 0x8fd5bb1 to 0xc84e85d 2024-09-16T14:08:43.434 INFO:teuthology.orchestra.run.smithi079.stdout:4324 writesame 0xa24ae3d thru 0xa24f89e data_size 0x1(0x4a62 bytes) 2024-09-16T14:08:43.454 INFO:teuthology.orchestra.run.smithi079.stdout:4325 write 0x3d2ecdc thru 0x3d355a1 (0x68c6 bytes) 2024-09-16T14:08:43.467 INFO:teuthology.orchestra.run.smithi079.stdout:4326 trunc from 0xc84e85d to 0xa073997 2024-09-16T14:08:43.474 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:43.472+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:43.487 INFO:teuthology.orchestra.run.smithi079.stdout:4327 punch from 0x1ac1343 to 0x1aca6b1, (0x936e bytes) 2024-09-16T14:08:43.488 INFO:teuthology.orchestra.run.smithi079.stdout:4328 write 0x7bae0c1 thru 0x7bba69f (0xc5df bytes) 2024-09-16T14:08:43.490 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.490 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.496 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.496 INFO:teuthology.orchestra.run.smithi079.stdout:4330 write 0x87c96d1 thru 0x87d78f9 (0xe229 bytes) 2024-09-16T14:08:43.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:43.504 INFO:teuthology.orchestra.run.smithi079.stdout:4331 write 0x683df76 thru 0x6840bd6 (0x2c61 bytes) 2024-09-16T14:08:43.512 INFO:teuthology.orchestra.run.smithi079.stdout:4332 read 0x22a5f59 thru 0x22b0d43 (0xadeb bytes) 2024-09-16T14:08:43.514 INFO:teuthology.orchestra.run.smithi079.stdout:4333 trunc from 0xa073997 to 0x91dd9ee 2024-09-16T14:08:43.543 INFO:teuthology.orchestra.run.smithi079.stdout:4335 write 0xbb70895 thru 0xbb7fc89 (0xf3f5 bytes) 2024-09-16T14:08:43.550 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.550 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.551 INFO:teuthology.orchestra.run.smithi079.stdout:4340 write 0x9bb0071 thru 0x9bb9a55 (0x99e5 bytes) 2024-09-16T14:08:43.575 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:43.570+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:43.578 INFO:teuthology.orchestra.run.smithi079.stdout:4342 trunc from 0xbb7fc8a to 0x1b85da5 2024-09-16T14:08:43.599 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.600 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.600 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.600 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.607 INFO:teuthology.orchestra.run.smithi079.stdout:4345 writesame 0x20f0b9d thru 0x20f9314 data_size 0x28(0x8778 bytes) 2024-09-16T14:08:43.609 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:43.627 INFO:teuthology.orchestra.run.smithi079.stdout:4346 writesame 0x7dff6e5 thru 0x7e00267 data_size 0x1(0xb83 bytes) 2024-09-16T14:08:43.628 INFO:teuthology.orchestra.run.smithi079.stdout:4347 trunc from 0x7e00268 to 0x90c9570 2024-09-16T14:08:43.639 INFO:teuthology.orchestra.run.smithi079.stdout:4350 write 0x742b56d thru 0x742e5ba (0x304e bytes) 2024-09-16T14:08:43.646 INFO:teuthology.orchestra.run.smithi079.stdout:4351 write 0x209ba54 thru 0x209c9ec (0xf99 bytes) 2024-09-16T14:08:43.654 INFO:teuthology.orchestra.run.smithi079.stdout:4352 write 0x3716b8b thru 0x371b036 (0x44ac bytes) 2024-09-16T14:08:43.661 INFO:teuthology.orchestra.run.smithi079.stdout:4356 read 0x9ec9e3 thru 0x9ee056 (0x1674 bytes) 2024-09-16T14:08:43.676 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:43.672+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:43.677 INFO:teuthology.orchestra.run.smithi079.stdout:4357 writesame 0xab6b6e5 thru 0xab6fa10 data_size 0x6(0x432c bytes) 2024-09-16T14:08:43.677 INFO:teuthology.orchestra.run.smithi079.stdout:4358 read 0x710603f thru 0x7114c61 (0xec23 bytes) 2024-09-16T14:08:43.677 INFO:teuthology.orchestra.run.smithi079.stdout:4362 writesame 0x866a4df thru 0x866c323 data_size 0x29(0x1e45 bytes) 2024-09-16T14:08:43.685 INFO:teuthology.orchestra.run.smithi079.stdout:4363 writesame 0x3ecb8a4 thru 0x3ecc3cc data_size 0x1(0xb29 bytes) 2024-09-16T14:08:43.693 INFO:teuthology.orchestra.run.smithi079.stdout:4365 write 0x8bf0818 thru 0x8bfca56 (0xc23f bytes) 2024-09-16T14:08:43.701 INFO:teuthology.orchestra.run.smithi079.stdout:4366 read 0x6341190 thru 0x6345a32 (0x48a3 bytes) 2024-09-16T14:08:43.701 INFO:teuthology.orchestra.run.smithi079.stdout:4367 writesame 0x80182c6 thru 0x801be27 data_size 0xb(0x3b62 bytes) 2024-09-16T14:08:43.703 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.703 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.713 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:43.725 INFO:teuthology.orchestra.run.smithi079.stdout:4369 trunc from 0xab6fa11 to 0x33ef554 2024-09-16T14:08:43.748 INFO:teuthology.orchestra.run.smithi079.stdout:4371 punch from 0x1483355 to 0x148ddde, (0xaa89 bytes) 2024-09-16T14:08:43.748 INFO:teuthology.orchestra.run.smithi079.stdout:4372 punch from 0x30f110b to 0x30f3889, (0x277e bytes) 2024-09-16T14:08:43.748 INFO:teuthology.orchestra.run.smithi079.stdout:4373 punch from 0x102c0e7 to 0x10338c9, (0x77e2 bytes) 2024-09-16T14:08:43.748 INFO:teuthology.orchestra.run.smithi079.stdout:4374 punch from 0xe27ca1 to 0xe2f825, (0x7b84 bytes) 2024-09-16T14:08:43.768 INFO:teuthology.orchestra.run.smithi079.stdout:4375 writesame 0x7043f7e thru 0x704ccad data_size 0x48(0x8d30 bytes) 2024-09-16T14:08:43.777 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:43.770+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:43.782 INFO:teuthology.orchestra.run.smithi079.stdout:4377 trunc from 0x704ccae to 0x39116c 2024-09-16T14:08:43.812 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.812 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.820 INFO:teuthology.orchestra.run.smithi079.stdout:4378 writesame 0x3503f89 thru 0x350ae42 data_size 0x1(0x6eba bytes) 2024-09-16T14:08:43.820 INFO:teuthology.orchestra.run.smithi079.stdout:4379 punch from 0x6064cd to 0x60cca1, (0x67d4 bytes) 2024-09-16T14:08:43.821 INFO:teuthology.orchestra.run.smithi079.stdout:4381 punch from 0x3065bc9 to 0x306d440, (0x7877 bytes) 2024-09-16T14:08:43.821 INFO:teuthology.orchestra.run.smithi079.stdout:4384 writesame 0x255050e thru 0x2557f0c data_size 0x1(0x79ff bytes) 2024-09-16T14:08:43.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:43.847 INFO:teuthology.orchestra.run.smithi079.stdout:4388 read 0x2b2f32f thru 0x2b2f839 (0x50b bytes) 2024-09-16T14:08:43.865 INFO:teuthology.orchestra.run.smithi079.stdout:4389 trunc from 0x350ae43 to 0xd8a0a56 2024-09-16T14:08:43.875 INFO:teuthology.orchestra.run.smithi079.stdout:4390 writesame 0xa3a8507 thru 0xa3a8535 data_size 0x1(0x2f bytes) 2024-09-16T14:08:43.877 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:43.876+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:43.881 INFO:teuthology.orchestra.run.smithi079.stdout:4392 writesame 0x386c8fe thru 0x3871d65 data_size 0x25(0x5468 bytes) 2024-09-16T14:08:43.888 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.901 INFO:teuthology.orchestra.run.smithi079.stdout:4396 write 0xe5f5d78 thru 0xe5f84fa (0x2783 bytes) 2024-09-16T14:08:43.917 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:43.917 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:43.920 INFO:teuthology.orchestra.run.smithi079.stdout:4397 trunc from 0xe5f84fb to 0x7e1e7a0 2024-09-16T14:08:43.927 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:43.940 INFO:teuthology.orchestra.run.smithi079.stdout:4398 writesame 0x26e4f8c thru 0x26e9b01 data_size 0xd(0x4b76 bytes) 2024-09-16T14:08:43.947 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.961 INFO:teuthology.orchestra.run.smithi079.stdout:4400 write 0x9cf1cdb thru 0x9cff759 (0xda7f bytes) 2024-09-16T14:08:43.968 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.968 INFO:teuthology.orchestra.run.smithi079.stdout:4403 write 0xe074c5 thru 0xe0ec87 (0x77c3 bytes) 2024-09-16T14:08:43.974 INFO:teuthology.orchestra.run.smithi079.stdout:4405 read 0x58b703c thru 0x58c0765 (0x972a bytes) 2024-09-16T14:08:43.974 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.974 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:43.979 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:43.976+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:43.994 INFO:teuthology.orchestra.run.smithi079.stdout:4409 write 0xd63d224 thru 0xd64cb15 (0xf8f2 bytes) 2024-09-16T14:08:44.004 INFO:teuthology.orchestra.run.smithi079.stdout:4411 punch from 0xae6e37b to 0xae7811c, (0x9da1 bytes) 2024-09-16T14:08:44.004 INFO:teuthology.orchestra.run.smithi079.stdout:4412 write 0xd435d2c thru 0xd43af52 (0x5227 bytes) 2024-09-16T14:08:44.010 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.010 INFO:teuthology.orchestra.run.smithi079.stdout:4414 read 0xa1636ac thru 0xa163921 (0x276 bytes) 2024-09-16T14:08:44.010 INFO:teuthology.orchestra.run.smithi079.stdout:4415 punch from 0xc80f61d to 0xc811d9b, (0x277e bytes) 2024-09-16T14:08:44.010 INFO:teuthology.orchestra.run.smithi079.stdout:4416 write 0x6439577 thru 0x644515a (0xbbe4 bytes) 2024-09-16T14:08:44.018 INFO:teuthology.orchestra.run.smithi079.stdout:4418 writesame 0x139480f thru 0x13983fd data_size 0x1(0x3bef bytes) 2024-09-16T14:08:44.024 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:44.025 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:44.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:44.046 INFO:teuthology.orchestra.run.smithi079.stdout:4420 write 0xe365178 thru 0xe36ee96 (0x9d1f bytes) 2024-09-16T14:08:44.053 INFO:teuthology.orchestra.run.smithi079.stdout:4421 writesame 0x1a8a3c0 thru 0x1a994cf data_size 0x98(0xf110 bytes) 2024-09-16T14:08:44.063 INFO:teuthology.orchestra.run.smithi079.stdout:4422 read 0x74cfccc thru 0x74de2ae (0xe5e3 bytes) 2024-09-16T14:08:44.063 INFO:teuthology.orchestra.run.smithi079.stdout:4423 punch from 0x31d8742 to 0x31daca6, (0x2564 bytes) 2024-09-16T14:08:44.063 INFO:teuthology.orchestra.run.smithi079.stdout:4424 writesame 0xe29d0e9 thru 0xe2a46df data_size 0x17(0x75f7 bytes) 2024-09-16T14:08:44.071 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.071 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.071 INFO:teuthology.orchestra.run.smithi079.stdout:4427 write 0x425af53 thru 0x4268d6d (0xde1b bytes) 2024-09-16T14:08:44.079 INFO:teuthology.orchestra.run.smithi079.stdout:4428 writesame 0x609616 thru 0x615150 data_size 0xd(0xbb3b bytes) 2024-09-16T14:08:44.079 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:44.076+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:44.084 INFO:teuthology.orchestra.run.smithi079.stdout:4429 write 0x9084a2e thru 0x908a813 (0x5de6 bytes) 2024-09-16T14:08:44.090 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.090 INFO:teuthology.orchestra.run.smithi079.stdout:4431 write 0x516903 thru 0x5220a9 (0xb7a7 bytes) 2024-09-16T14:08:44.093 INFO:teuthology.orchestra.run.smithi079.stdout:4432 punch from 0x6987c99 to 0x6993cf0, (0xc057 bytes) 2024-09-16T14:08:44.109 INFO:teuthology.orchestra.run.smithi079.stdout:4433 trunc from 0xe36ee97 to 0x4cbac8c 2024-09-16T14:08:44.131 INFO:teuthology.orchestra.run.smithi079.stdout:4434 read 0x22e4546 thru 0x22e5bd5 (0x1690 bytes) 2024-09-16T14:08:44.131 INFO:teuthology.orchestra.run.smithi079.stdout:4436 read 0x300484 thru 0x30aba7 (0xa724 bytes) 2024-09-16T14:08:44.131 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.133 INFO:teuthology.orchestra.run.smithi079.stdout:4439 trunc from 0x4cbac8c to 0x63fcc11 2024-09-16T14:08:44.154 INFO:teuthology.orchestra.run.smithi079.stdout:4440 write 0x76dc3c4 thru 0x76eaf92 (0xebcf bytes) 2024-09-16T14:08:44.155 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:44.155 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:44.163 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.163 INFO:teuthology.orchestra.run.smithi079.stdout:4442 punch from 0x4c8ad5a to 0x4c929e6, (0x7c8c bytes) 2024-09-16T14:08:44.164 INFO:teuthology.orchestra.run.smithi079.stdout:4443 read 0x4e32669 thru 0x4e32cb7 (0x64f bytes) 2024-09-16T14:08:44.164 INFO:teuthology.orchestra.run.smithi079.stdout:4444 read 0x4c9e2e1 thru 0x4c9f988 (0x16a8 bytes) 2024-09-16T14:08:44.164 INFO:teuthology.orchestra.run.smithi079.stdout:4445 trunc from 0x76eaf93 to 0x70f7559 2024-09-16T14:08:44.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:44.180 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:44.174+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:44.183 INFO:teuthology.orchestra.run.smithi079.stdout:4447 read 0x6554d43 thru 0x6561d58 (0xd016 bytes) 2024-09-16T14:08:44.183 INFO:teuthology.orchestra.run.smithi079.stdout:4448 punch from 0x797afa to 0x7a0a80, (0x8f86 bytes) 2024-09-16T14:08:44.183 INFO:teuthology.orchestra.run.smithi079.stdout:4450 write 0x8a3c52 thru 0x8b361e (0xf9cd bytes) 2024-09-16T14:08:44.187 INFO:teuthology.orchestra.run.smithi079.stdout:4451 punch from 0x295fc0d to 0x296fa08, (0xfdfb bytes) 2024-09-16T14:08:44.187 INFO:teuthology.orchestra.run.smithi079.stdout:4453 read 0x43b673a thru 0x43bf54e (0x8e15 bytes) 2024-09-16T14:08:44.212 INFO:teuthology.orchestra.run.smithi079.stdout:4454 write 0xdec7f56 thru 0xdecc70b (0x47b6 bytes) 2024-09-16T14:08:44.219 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.230 INFO:teuthology.orchestra.run.smithi079.stdout:4456 write 0xe7c9ce7 thru 0xe7d4352 (0xa66c bytes) 2024-09-16T14:08:44.237 INFO:teuthology.orchestra.run.smithi079.stdout:4459 writesame 0xa3d9f0 thru 0xa43500 data_size 0x3(0x5b11 bytes) 2024-09-16T14:08:44.248 INFO:teuthology.orchestra.run.smithi079.stdout:4460 writesame 0xc607f70 thru 0xc613479 data_size 0x1(0xb50a bytes) 2024-09-16T14:08:44.277 INFO:teuthology.orchestra.run.smithi079.stdout:4464 read 0x194acb2 thru 0x19581bf (0xd50e bytes) 2024-09-16T14:08:44.277 INFO:teuthology.orchestra.run.smithi079.stdout:4465 writesame 0xc4ee518 thru 0xc4fa138 data_size 0x1(0xbc21 bytes) 2024-09-16T14:08:44.281 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:44.274+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:44.286 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:44.286 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:44.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:44.308 INFO:teuthology.orchestra.run.smithi079.stdout:4466 writesame 0x75e6fc thru 0x76aad9 data_size 0x3d(0xc3de bytes) 2024-09-16T14:08:44.311 INFO:teuthology.orchestra.run.smithi079.stdout:4467 write 0x94ef0c1 thru 0x94f94fa (0xa43a bytes) 2024-09-16T14:08:44.317 INFO:teuthology.orchestra.run.smithi079.stdout:4468 write 0x9bb4c9 thru 0x9c0533 (0x506b bytes) 2024-09-16T14:08:44.324 INFO:teuthology.orchestra.run.smithi079.stdout:4469 punch from 0x35ea06a to 0x35edeba, (0x3e50 bytes) 2024-09-16T14:08:44.324 INFO:teuthology.orchestra.run.smithi079.stdout:4470 write 0x1d42c1c thru 0x1d5022e (0xd613 bytes) 2024-09-16T14:08:44.333 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.333 INFO:teuthology.orchestra.run.smithi079.stdout:4472 read 0xd72f7de thru 0xd736fef (0x7812 bytes) 2024-09-16T14:08:44.333 INFO:teuthology.orchestra.run.smithi079.stdout:4476 read 0xb82390c thru 0xb827069 (0x375e bytes) 2024-09-16T14:08:44.333 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.333 INFO:teuthology.orchestra.run.smithi079.stdout:4480 writesame 0x7140849 thru 0x714aacb data_size 0x1(0xa283 bytes) 2024-09-16T14:08:44.363 INFO:teuthology.orchestra.run.smithi079.stdout:4481 read 0x2a871b2 thru 0x2a88bd5 (0x1a24 bytes) 2024-09-16T14:08:44.364 INFO:teuthology.orchestra.run.smithi079.stdout:4482 write 0x56fb24e thru 0x57029be (0x7771 bytes) 2024-09-16T14:08:44.375 INFO:teuthology.orchestra.run.smithi079.stdout:4483 trunc from 0xe7d4353 to 0xcaa44ea 2024-09-16T14:08:44.382 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:44.374+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:44.395 INFO:teuthology.orchestra.run.smithi079.stdout:4484 read 0x739be8 thru 0x742a60 (0x8e79 bytes) 2024-09-16T14:08:44.396 INFO:teuthology.orchestra.run.smithi079.stdout:4485 read 0x2938bd1 thru 0x2938c97 (0xc7 bytes) 2024-09-16T14:08:44.396 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.396 INFO:teuthology.orchestra.run.smithi079.stdout:4487 punch from 0x8bdc521 to 0x8be351c, (0x6ffb bytes) 2024-09-16T14:08:44.396 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.396 INFO:teuthology.orchestra.run.smithi079.stdout:4489 punch from 0xefff93 to 0xf010d8, (0x1145 bytes) 2024-09-16T14:08:44.402 INFO:teuthology.orchestra.run.smithi079.stdout:4490 trunc from 0xcaa44ea to 0x8f490e2 2024-09-16T14:08:44.417 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:44.417 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:44.423 INFO:teuthology.orchestra.run.smithi079.stdout:4492 write 0x1a239c8 thru 0x1a2a009 (0x6642 bytes) 2024-09-16T14:08:44.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:44.435 INFO:teuthology.orchestra.run.smithi079.stdout:4493 trunc from 0x8f490e2 to 0xc185ba5 2024-09-16T14:08:44.444 INFO:teuthology.orchestra.run.smithi079.stdout:4499 read 0x623e605 thru 0x624c7eb (0xe1e7 bytes) 2024-09-16T14:08:44.444 INFO:teuthology.orchestra.run.smithi079.stdout:4500 write 0x3e366a5 thru 0x3e3bbd7 (0x5533 bytes) 2024-09-16T14:08:44.450 INFO:teuthology.orchestra.run.smithi079.stdout:4501 punch from 0x367937c to 0x3679b32, (0x7b6 bytes) 2024-09-16T14:08:44.462 INFO:teuthology.orchestra.run.smithi079.stdout:4502 writesame 0xd3e2419 thru 0xd3ef1e6 data_size 0x9(0xcdce bytes) 2024-09-16T14:08:44.462 INFO:teuthology.orchestra.run.smithi079.stdout:4503 write 0x6cdd396 thru 0x6ce6e4d (0x9ab8 bytes) 2024-09-16T14:08:44.480 INFO:teuthology.orchestra.run.smithi079.stdout:4506 writesame 0xd936f71 thru 0xd93f28f data_size 0x3(0x831f bytes) 2024-09-16T14:08:44.483 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:44.478+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:44.492 INFO:teuthology.orchestra.run.smithi079.stdout:4507 writesame 0xeda1fc7 thru 0xeda4a74 data_size 0x9(0x2aae bytes) 2024-09-16T14:08:44.492 INFO:teuthology.orchestra.run.smithi079.stdout:4509 read 0x906d3df thru 0x907676e (0x9390 bytes) 2024-09-16T14:08:44.492 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:44.492 INFO:teuthology.orchestra.run.smithi079.stdout:4511 read 0x46ced76 thru 0x46d4d75 (0x6000 bytes) 2024-09-16T14:08:44.493 INFO:teuthology.orchestra.run.smithi079.stdout:4512 write 0x6ae349b thru 0x6aeb3fa (0x7f60 bytes) 2024-09-16T14:08:44.500 INFO:teuthology.orchestra.run.smithi079.stdout:4513 clone 37 order 18 su 131072 sc 3 2024-09-16T14:08:44.536 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:44.536 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:44.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:44.584 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:44.578+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:44.685 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:44.680+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:44.686 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:44.687 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:44.696 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:44.785 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:44.780+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:44.793 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:44.794 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:44.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:44.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:44.886 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:44.884+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:44.946 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:44.946 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:44.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:44.978 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:44.987 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:44.984+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:44.998 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:45.015 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:45.034 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:45.046 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:45.046 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:45.046 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:45.046 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:45.046 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:45.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:45.057 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:45.079 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:45.088 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:45.082+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:45.099 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:45.121 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:45.123 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:45.157 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.250936136, 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:08:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:08:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.250913427999997, 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:08:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:08:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.250889276000002, 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:08:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:08:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:08:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:08:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.250856335000002, 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.250831652000002, 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:08:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.250808423000002, 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.248962458999998, 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:08:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.248919594999997, 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:08:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247459976999998, 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:08:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247444207999997, 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:08:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:08:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247428456000002, 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:08:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:08:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247413025, 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247397386999999, 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:08:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247382248000001, 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247367410999999, 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:08:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:08:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247344407, 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247329182000001, 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:08:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:08:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247313998000003, 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:08:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247298753000003, 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.247283594999999, 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:08:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.272 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:45.273 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:45.274 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:45.266+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:45.374 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:45.372+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:45.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:08:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:45.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:45.475 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:45.470+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:45.576 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:45.572+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:45.586 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:45.586 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:45.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:45.669 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-16T14:08:45.677 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:45.672+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:45.687 INFO:teuthology.orchestra.run.smithi079.stdout:4515 punch from 0xa3befb8 to 0xa3cde0e, (0xee56 bytes) 2024-09-16T14:08:45.694 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:45.694 INFO:teuthology.orchestra.run.smithi079.stdout:4518 writesame 0xe2e1515 thru 0xe2e23f4 data_size 0x11(0xee0 bytes) 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86627 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 115712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.521073633, 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057990535000000003, 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86627 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172072+0000", 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.79e-07 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172073+0000", 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.53e-06 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172077+0000", 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3330000000000001e-06 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.172079+0000", 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224325+0000", 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.052246133 2024-09-16T14:08:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.224394+0000", 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8960000000000004e-05 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.230062+0000", 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056682820000000002 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86989 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.499464787000001, 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058737710999999998, 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86989 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193680+0000", 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193681+0000", 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193682+0000", 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6169999999999999e-06 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193687+0000", 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7210000000000002e-06 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193690+0000", 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.072e-06 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.246881+0000", 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.053191117000000003 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252309+0000", 2024-09-16T14:08:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054275979999999996 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.252418+0000", 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000109255 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:86995 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 183296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.499438585, 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059640662999999997, 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 86995 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.193707+0000", 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197498+0000", 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037915219999999999 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197504+0000", 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7520000000000002e-06 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197515+0000", 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.117e-05 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197521+0000", 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.083e-06 2024-09-16T14:08:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253121+0000", 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055600254000000002 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253227+0000", 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010607000000000001 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.253347+0000", 2024-09-16T14:08:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119812 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87017 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 192512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495502848999998, 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676182999999998, 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87017 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197642+0000", 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197643+0000", 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197645+0000", 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197652+0000", 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8430000000000004e-06 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197656+0000", 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254046+0000", 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056390033999999999 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254205+0000", 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000158639 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254318+0000", 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113916 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87019 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 229376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495485273, 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056580459, 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87019 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197660+0000", 2024-09-16T14:08:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3000000000000001e-07 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197663+0000", 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.396e-06 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197669+0000", 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4099999999999996e-06 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197673+0000", 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254066+0000", 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056392870999999997 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254160+0000", 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4574000000000006e-05 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254240+0000", 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0125999999999998e-05 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87023 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 193536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495468406000001, 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056736048999999997, 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87023 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197677+0000", 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4200000000000001e-07 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197679+0000", 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0899999999999999e-06 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197686+0000", 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3219999999999998e-06 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197690+0000", 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.177e-06 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254263+0000", 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056573364000000001 2024-09-16T14:08:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254334+0000", 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0574000000000006e-05 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254413+0000", 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9079999999999995e-05 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87027 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495451283000001, 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056974745, 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87027 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197694+0000", 2024-09-16T14:08:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2599999999999998e-07 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197696+0000", 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1679999999999998e-06 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197703+0000", 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4389999999999997e-06 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197707+0000", 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7060000000000002e-06 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254383+0000", 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056676509999999999 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254483+0000", 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9692999999999996e-05 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254669+0000", 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018580300000000001 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87029 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 197632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495434445999997, 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057332283999999997, 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87029 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197711+0000", 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197713+0000", 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.199e-06 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197719+0000", 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.699e-06 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197723+0000", 2024-09-16T14:08:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8700000000000002e-06 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254709+0000", 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056986536999999997 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254790+0000", 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0192999999999996e-05 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255043+0000", 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025339500000000001 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87031 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 234496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495418153999999, 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057120535, 2024-09-16T14:08:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87031 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197727+0000", 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0600000000000001e-07 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197729+0000", 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0499999999999999e-06 2024-09-16T14:08:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197736+0000", 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2249999999999997e-06 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197739+0000", 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6839999999999998e-06 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254697+0000", 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056957902999999997 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254776+0000", 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9221999999999995e-05 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254848+0000", 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1044999999999995e-05 2024-09-16T14:08:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87035 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 198656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495402149999997, 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057340290000000002, 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87035 2024-09-16T14:08:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197743+0000", 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197744+0000", 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6499999999999999e-07 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197746+0000", 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0169999999999999e-06 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197752+0000", 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1040000000000003e-06 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197755+0000", 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.782e-06 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254869+0000", 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057113766000000003 2024-09-16T14:08:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.254971+0000", 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101932 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255083+0000", 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112124 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87039 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 238592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495385740000003, 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057586228000000003, 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87039 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197759+0000", 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197760+0000", 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197762+0000", 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.829e-06 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197768+0000", 2024-09-16T14:08:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5470000000000002e-06 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197772+0000", 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4699999999999998e-06 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255109+0000", 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057337232000000002 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255184+0000", 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5593000000000006e-05 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255346+0000", 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016120899999999999 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87041 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 202752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495369685999997, 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057827176000000001, 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87041 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197775+0000", 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197776+0000", 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197778+0000", 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3709999999999998e-06 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197784+0000", 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.054e-06 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197787+0000", 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1879999999999998e-06 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255397+0000", 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057609059999999997 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255483+0000", 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6415999999999994e-05 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255603+0000", 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119703 2024-09-16T14:08:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87043 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 203776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495353518999998, 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057723202000000001, 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87043 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197792+0000", 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197801+0000", 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5869999999999993e-06 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197807+0000", 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4160000000000001e-06 2024-09-16T14:08:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197811+0000", 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0099999999999997e-06 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255377+0000", 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057566063000000001 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255450+0000", 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3192e-05 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255515+0000", 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4543999999999997e-05 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87045 3.e 3:715d7993:::rbd_data.113619b3ee73.000000000000003a:16 [read 239616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495329325999997, 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057864405000000001, 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87045 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197816+0000", 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197819+0000", 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.311e-06 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197824+0000", 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6980000000000003e-06 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197828+0000", 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2339999999999999e-06 2024-09-16T14:08:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255537+0000", 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057709979000000002 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255613+0000", 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5807000000000001e-05 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255680+0000", 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6935999999999997e-05 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87051 3.16 3:6bc03878:::rbd_data.113619b3ee73.0000000000000032:16 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.495314174000001, 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058010645999999999, 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87051 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.197831+0000", 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200527+0000", 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026964990000000002 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200532+0000", 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8899999999999998e-06 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200541+0000", 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1249999999999993e-06 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200546+0000", 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4129999999999997e-06 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255630+0000", 2024-09-16T14:08:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055083612999999997 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255711+0000", 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1934000000000005e-05 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.255842+0000", 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013017200000000001 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87559 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 100352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.492479818, 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056509688000000002, 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87559 2024-09-16T14:08:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200665+0000", 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200666+0000", 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200668+0000", 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.21e-06 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200674+0000", 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.896e-06 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200678+0000", 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7050000000000001e-06 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.256898+0000", 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056220883999999999 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257014+0000", 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011530499999999999 2024-09-16T14:08:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257175+0000", 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016127200000000001 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87568 3.16 3:69d74893:::rbd_data.113619b3ee73.0000000000000026:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.492447839, 2024-09-16T14:08:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056739408999999998, 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87568 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200697+0000", 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200698+0000", 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200700+0000", 2024-09-16T14:08:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.272e-06 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200706+0000", 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7080000000000002e-06 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200709+0000", 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.394e-06 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257221+0000", 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056512383999999999 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257319+0000", 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7510000000000007e-05 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257437+0000", 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011773399999999999 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:87570 3.e 3:73b2712a:::rbd_data.113619b3ee73.0000000000000027:16 [read 105472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.492432506999997, 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.056717201000000002, 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87570 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.200713+0000", 2024-09-16T14:08:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201471+0000", 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075828899999999999 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201480+0000", 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.448e-06 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201494+0000", 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3162999999999999e-05 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.201500+0000", 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0090000000000004e-06 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257205+0000", 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.055705603999999999 2024-09-16T14:08:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257330+0000", 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000124972 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:59.257430+0000", 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9716000000000005e-05 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.174247349, 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:08:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.1689760099999997, 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:08:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:08:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:08:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:45.793 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:45.794 INFO:teuthology.orchestra.run.smithi079.stdout:4519 trunc from 0xeda4a75 to 0x930d4e9 2024-09-16T14:08:45.795 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:45.796 INFO:teuthology.orchestra.run.smithi079.stdout:4520 read 0x567100e thru 0x567de0c (0xcdff bytes) 2024-09-16T14:08:45.796 INFO:teuthology.orchestra.run.smithi079.stdout:4521 trunc from 0x930d4e9 to 0x33f4c4f 2024-09-16T14:08:45.796 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:45.790+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:45.797 INFO:teuthology.orchestra.run.smithi079.stdout:4522 write 0xab5bc89 thru 0xab63d4a (0x80c2 bytes) 2024-09-16T14:08:45.810 INFO:teuthology.orchestra.run.smithi079.stdout:4525 punch from 0x5048275 to 0x504e6b4, (0x643f bytes) 2024-09-16T14:08:45.810 INFO:teuthology.orchestra.run.smithi079.stdout:4526 read 0x56a6a03 thru 0x56b40cf (0xd6cd bytes) 2024-09-16T14:08:45.810 INFO:teuthology.orchestra.run.smithi079.stdout:4527 write 0x7ae3a62 thru 0x7aed076 (0x9615 bytes) 2024-09-16T14:08:45.817 INFO:teuthology.orchestra.run.smithi079.stdout:4528 punch from 0x71cdb32 to 0x71d57b1, (0x7c7f bytes) 2024-09-16T14:08:45.817 INFO:teuthology.orchestra.run.smithi079.stdout:4530 write 0x46f784c thru 0x46fbbee (0x43a3 bytes) 2024-09-16T14:08:45.824 INFO:teuthology.orchestra.run.smithi079.stdout:4531 write 0x62a9fc thru 0x63756c (0xcb71 bytes) 2024-09-16T14:08:45.833 INFO:teuthology.orchestra.run.smithi079.stdout:4532 flatten 2024-09-16T14:08:45.895 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:45.892+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:45.898 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:45.912 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:45.912 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:45.912 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:45.913 INFO:teuthology.orchestra.run.smithi079.stdout:4534 writesame 0xe44a689 thru 0xe457d25 data_size 0x1(0xd69d bytes) 2024-09-16T14:08:45.913 INFO:teuthology.orchestra.run.smithi079.stdout:4535 punch from 0xe41ada8 to 0xe42a3cf, (0xf627 bytes) 2024-09-16T14:08:45.913 INFO:teuthology.orchestra.run.smithi079.stdout:4536 writesame 0x92521be thru 0x9261e8f data_size 0xa1(0xfcd2 bytes) 2024-09-16T14:08:45.922 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:45.923 INFO:teuthology.orchestra.run.smithi079.stdout:4537 read 0x306b554 thru 0x3073860 (0x830d bytes) 2024-09-16T14:08:45.923 INFO:teuthology.orchestra.run.smithi079.stdout:4540 write 0x247c78d thru 0x248bfa1 (0xf815 bytes) 2024-09-16T14:08:45.930 INFO:teuthology.orchestra.run.smithi079.stdout:4541 read 0xaf936d1 thru 0xaf9e4e0 (0xae10 bytes) 2024-09-16T14:08:45.930 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:45.930 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:45.930 INFO:teuthology.orchestra.run.smithi079.stdout:4545 write 0x8a6e5cd thru 0x8a7d668 (0xf09c bytes) 2024-09-16T14:08:45.937 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:45.937 INFO:teuthology.orchestra.run.smithi079.stdout:4548 read 0xba75fec thru 0xba7e849 (0x885e bytes) 2024-09-16T14:08:45.937 INFO:teuthology.orchestra.run.smithi079.stdout:4549 punch from 0x8d5e3a4 to 0x8d654bb, (0x7117 bytes) 2024-09-16T14:08:45.937 INFO:teuthology.orchestra.run.smithi079.stdout:4552 writesame 0x295100f thru 0x295ff5b data_size 0x1(0xef4d bytes) 2024-09-16T14:08:45.979 INFO:teuthology.orchestra.run.smithi079.stdout:4553 punch from 0x29f8125 to 0x29f8cc8, (0xba3 bytes) 2024-09-16T14:08:45.979 INFO:teuthology.orchestra.run.smithi079.stdout:4555 write 0xaf8e6f3 thru 0xaf97768 (0x9076 bytes) 2024-09-16T14:08:45.986 INFO:teuthology.orchestra.run.smithi079.stdout:4556 read 0xa3b20f8 thru 0xa3bcab2 (0xa9bb bytes) 2024-09-16T14:08:45.986 INFO:teuthology.orchestra.run.smithi079.stdout:4557 write 0x99e994a thru 0x99f0e44 (0x74fb bytes) 2024-09-16T14:08:45.994 INFO:teuthology.orchestra.run.smithi079.stdout:4559 punch from 0xaeb7e32 to 0xaebb819, (0x39e7 bytes) 2024-09-16T14:08:45.994 INFO:teuthology.orchestra.run.smithi079.stdout:4560 write 0x8c63c41 thru 0x8c649b3 (0xd73 bytes) 2024-09-16T14:08:45.996 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:45.992+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:46.000 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:46.000 INFO:teuthology.orchestra.run.smithi079.stdout:4565 writesame 0x81120e5 thru 0x811a5ca data_size 0x1(0x84e6 bytes) 2024-09-16T14:08:46.023 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:46.023 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:46.023 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:46.023 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:46.023 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:46.026 INFO:teuthology.orchestra.run.smithi079.stdout:4566 writesame 0xa702340 thru 0xa70640d data_size 0x4f(0x40ce bytes) 2024-09-16T14:08:46.033 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:46.034 INFO:teuthology.orchestra.run.smithi079.stdout:4567 clone 38 order 24 su 262144 sc 6 2024-09-16T14:08:46.097 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:46.092+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.789566887000007, 2024-09-16T14:08:46.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:46.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.789544489999997, 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:46.137 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:46.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:46.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.788937778999994, 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:46.143 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:46.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:46.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:46.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.788863452000001, 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:46.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:46.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:46.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.817938659999996, 2024-09-16T14:08:46.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:46.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.189980078999994, 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:46.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:46.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:46.166 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.659819612999996, 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:46.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:46.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:46.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.012271081999998, 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:46.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.465980842, 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:46.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.465963651000003, 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:46.182 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:46.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.465926633999999, 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.187 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:46.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.465905958, 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:46.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:46.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.196 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.465863777999999, 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:46.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.465829552999999, 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.201 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:46.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.205 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.465799836999999, 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:46.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:46.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.465780174999999, 2024-09-16T14:08:46.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:46.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:46.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.457020049999997, 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.215 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:46.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.372293507999998, 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:46.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7272899710000003, 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:08:46.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:08:46.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:08:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:08:46.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7272031099999996, 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:08:46.230 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:08:46.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:46.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.237 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.237 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:46.238 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:46.239 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:46.234+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:46.339 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:46.334+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:46.341 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:46.342 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:46.342 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:46.351 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:08:46.450 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:46.448+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:46.454 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:46.454 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:46.454 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:46.454 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:46.454 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:46.463 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:46.541 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:46.534+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:46.562 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.728178678999999, 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.728162340000001, 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:08:46.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:08:46.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.728137175000001, 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:08:46.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:08:46.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.727233208000001, 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.582 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:08:46.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.727223868999999, 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.587 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:08:46.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:08:46.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.592 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.726933747, 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:08:46.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:08:46.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:08:46.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.726715053, 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:46.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:08:46.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.603 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.720321473999999, 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:08:46.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:08:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.716409911, 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.609 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:08:46.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:46.614 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.716378706, 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:46.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:08:46.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.715559266, 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:08:46.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.715514457000001, 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:08:46.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:08:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.712505202999999, 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:08:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:08:46.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:08:46.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.712488242999999, 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:08:46.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:08:46.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:08:46.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.712476747, 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.707729836999999, 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:08:46.645 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:08:46.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:08:46.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.700097205999999, 2024-09-16T14:08:46.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:08:46.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.700084999000001, 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:08:46.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:08:46.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:08:46.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.697830084, 2024-09-16T14:08:46.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:08:46.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:08:46.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.697605833000001, 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:46.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:08:46.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:46.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:08:46.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:08:46.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.672 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:46.672 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:46.672 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:46.674 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:46.674 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:46.668+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:46.681 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-16T14:08:46.775 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:46.772+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:46.781 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:46.781 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:46.791 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:46.875 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:46.870+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:46.890 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:46.890 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:46.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:46.977 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:46.972+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:47.001 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.623054891999999, 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:08:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:08:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.619123473999998, 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:08:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.619099282000001, 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:08:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.615380010999999, 2024-09-16T14:08:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:08:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.611463246, 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:08:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.607080967000002, 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:08:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.607057126000001, 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:08:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.607041938000002, 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:08:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.607027352000003, 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:08:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.606923702000003, 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:08:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:08:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.606895975999997, 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:08:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.604222360000001, 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:08:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:08:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.604195646000001, 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:08:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.604155478999999, 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:08:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.604065763999998, 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:08:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.603656960000002, 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:08:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:08:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.603556775999998, 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.602658138000002, 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:08:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:08:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.595459406000003, 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:08:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.593291114000003, 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:08:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:47.099 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:47.100 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:47.101 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:47.096+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:47.106 INFO:teuthology.orchestra.run.smithi079.stdout:4568 writesame 0xa13d25b thru 0xa140501 data_size 0x1(0x32a7 bytes) 2024-09-16T14:08:47.131 INFO:teuthology.orchestra.run.smithi079.stdout:4569 punch from 0xdb4b5d9 to 0xdb52a1f, (0x7446 bytes) 2024-09-16T14:08:47.131 INFO:teuthology.orchestra.run.smithi079.stdout:4570 punch from 0x8f893de to 0x8f97e9d, (0xeabf bytes) 2024-09-16T14:08:47.131 INFO:teuthology.orchestra.run.smithi079.stdout:4571 write 0x57116c1 thru 0x5718643 (0x6f83 bytes) 2024-09-16T14:08:47.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:47.144 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.144 INFO:teuthology.orchestra.run.smithi079.stdout:4573 write 0x538c977 thru 0x53909e4 (0x406e bytes) 2024-09-16T14:08:47.161 INFO:teuthology.orchestra.run.smithi079.stdout:4574 trunc from 0xe457d26 to 0xb0f2d49 2024-09-16T14:08:47.181 INFO:teuthology.orchestra.run.smithi079.stdout:4576 write 0x20514f9 thru 0x2057306 (0x5e0e bytes) 2024-09-16T14:08:47.191 INFO:teuthology.orchestra.run.smithi079.stdout:4577 write 0x3cf70c thru 0x3d9d27 (0xa61c bytes) 2024-09-16T14:08:47.194 INFO:teuthology.orchestra.run.smithi079.stdout:4579 read 0x7516d68 thru 0x751bd1c (0x4fb5 bytes) 2024-09-16T14:08:47.195 INFO:teuthology.orchestra.run.smithi079.stdout:4580 writesame 0x909f8c2 thru 0x90a16ae data_size 0x1(0x1ded bytes) 2024-09-16T14:08:47.201 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:47.196+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:47.208 INFO:teuthology.orchestra.run.smithi079.stdout:4581 trunc from 0xb0f2d49 to 0xc4c6d2d 2024-09-16T14:08:47.210 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:47.210 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:47.210 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:47.218 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.219 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:47.237 INFO:teuthology.orchestra.run.smithi079.stdout:4583 trunc from 0xc4c6d2d to 0xb39359 2024-09-16T14:08:47.265 INFO:teuthology.orchestra.run.smithi079.stdout:4584 read 0xad2fdc thru 0xad37bd (0x7e2 bytes) 2024-09-16T14:08:47.266 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.266 INFO:teuthology.orchestra.run.smithi079.stdout:4587 read 0x1b0dc0 thru 0x1b1c02 (0xe43 bytes) 2024-09-16T14:08:47.267 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.267 INFO:teuthology.orchestra.run.smithi079.stdout:4591 punch from 0x1fa00d to 0x202489, (0x847c bytes) 2024-09-16T14:08:47.267 INFO:teuthology.orchestra.run.smithi079.stdout:4592 punch from 0x3a8733 to 0x3ae235, (0x5b02 bytes) 2024-09-16T14:08:47.273 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:47.284 INFO:teuthology.orchestra.run.smithi079.stdout:4593 trunc from 0xb39359 to 0xc08e898 2024-09-16T14:08:47.294 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:47.301 INFO:teuthology.orchestra.run.smithi079.stdout:4596 write 0xcd9396b thru 0xcd944e9 (0xb7f bytes) 2024-09-16T14:08:47.301 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:47.294+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:47.305 INFO:teuthology.orchestra.run.smithi079.stdout:4599 read 0x539be73 thru 0x539c06a (0x1f8 bytes) 2024-09-16T14:08:47.306 INFO:teuthology.orchestra.run.smithi079.stdout:4600 read 0xc2e7887 thru 0xc2e7cd6 (0x450 bytes) 2024-09-16T14:08:47.306 INFO:teuthology.orchestra.run.smithi079.stdout:4601 punch from 0x9261ed3 to 0x926974f, (0x787c bytes) 2024-09-16T14:08:47.306 INFO:teuthology.orchestra.run.smithi079.stdout:4602 write 0x9822ec7 thru 0x982f523 (0xc65d bytes) 2024-09-16T14:08:47.319 INFO:teuthology.orchestra.run.smithi079.stdout:4603 trunc from 0xcd944ea to 0x8c6d661 2024-09-16T14:08:47.319 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:47.325 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:47.325 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:47.334 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:47.337 INFO:teuthology.orchestra.run.smithi079.stdout:4604 write 0x441ffaf thru 0x442d860 (0xd8b2 bytes) 2024-09-16T14:08:47.340 INFO:teuthology.orchestra.run.smithi079.stdout:4605 punch from 0x68459 to 0x68e11, (0x9b8 bytes) 2024-09-16T14:08:47.340 INFO:teuthology.orchestra.run.smithi079.stdout:4608 writesame 0x4b75b05 thru 0x4b83040 data_size 0x6(0xd53c bytes) 2024-09-16T14:08:47.343 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:47.347 INFO:teuthology.orchestra.run.smithi079.stdout:4611 write 0x262572e thru 0x262c242 (0x6b15 bytes) 2024-09-16T14:08:47.350 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.350 INFO:teuthology.orchestra.run.smithi079.stdout:4613 read 0xce3fa4 thru 0xcf2874 (0xe8d1 bytes) 2024-09-16T14:08:47.351 INFO:teuthology.orchestra.run.smithi079.stdout:4614 writesame 0x489b15a thru 0x489e210 data_size 0x1(0x30b7 bytes) 2024-09-16T14:08:47.361 INFO:teuthology.orchestra.run.smithi079.stdout:4615 write 0x43cb393 thru 0x43d9b3f (0xe7ad bytes) 2024-09-16T14:08:47.366 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:47.373 INFO:teuthology.orchestra.run.smithi079.stdout:4616 trunc from 0x8c6d661 to 0x2b0c66d 2024-09-16T14:08:47.392 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:47.402 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:47.398+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:47.412 INFO:teuthology.orchestra.run.smithi079.stdout:4618 trunc from 0x2b0c66d to 0xe9e2eea 2024-09-16T14:08:47.415 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:47.419 INFO:teuthology.orchestra.run.smithi079.stdout:4619 read 0x14540e1 thru 0x145e37c (0xa29c bytes) 2024-09-16T14:08:47.420 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.420 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.420 INFO:teuthology.orchestra.run.smithi079.stdout:4623 trunc from 0xe9e2eea to 0xec3335e 2024-09-16T14:08:47.429 INFO:teuthology.orchestra.run.smithi079.stdout:4624 write 0xb2a68b7 thru 0xb2b3767 (0xceb1 bytes) 2024-09-16T14:08:47.435 INFO:teuthology.orchestra.run.smithi079.stdout:4625 punch from 0x1ed1131 to 0x1ed1f23, (0xdf2 bytes) 2024-09-16T14:08:47.435 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.435 INFO:teuthology.orchestra.run.smithi079.stdout:4627 punch from 0x15b49b6 to 0x15b8fe1, (0x462b bytes) 2024-09-16T14:08:47.435 INFO:teuthology.orchestra.run.smithi079.stdout:4628 write 0x7e7d8ba thru 0x7e8bdbf (0xe506 bytes) 2024-09-16T14:08:47.438 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:47.440 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:47.443 INFO:teuthology.orchestra.run.smithi079.stdout:4631 punch from 0x4cec99a to 0x4ced77d, (0xde3 bytes) 2024-09-16T14:08:47.443 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:47.443 INFO:teuthology.orchestra.run.smithi079.stdout:4634 writesame 0xbd6a412 thru 0xbd72f47 data_size 0x67(0x8b36 bytes) 2024-09-16T14:08:47.460 INFO:teuthology.orchestra.run.smithi079.stdout:4635 trunc from 0xec3335e to 0x6ccbfad 2024-09-16T14:08:47.476 INFO:teuthology.orchestra.run.smithi079.stdout:4636 trunc from 0x6ccbfad to 0x6173b2a 2024-09-16T14:08:47.481 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:47.482 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:47.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:47.511 INFO:teuthology.orchestra.run.smithi079.stdout:4637 writesame 0xb5131dd thru 0xb521de0 data_size 0x72(0xec04 bytes) 2024-09-16T14:08:47.511 INFO:teuthology.orchestra.run.smithi079.stdout:4638 clone 39 order 19 su 4096 sc 2 2024-09-16T14:08:47.511 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:47.508+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:47.596 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:47.596 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:47.604 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:47.598+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:47.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:47.705 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:47.700+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:47.759 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:47.759 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:47.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:47.805 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:47.798+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:47.816 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:08:47.816 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T14:08:47.871 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:47.871 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:47.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:47.907 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:47.902+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:47.996 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:47.996 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:48.009 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:48.004+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:48.102 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.102 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.110 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:48.104+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:48.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:48.206 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.206 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.211 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:48.208+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:48.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:48.309 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.309 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.311 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:48.306+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:48.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:48.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:08:48.412 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:48.406+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:48.418 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T14:08:48.423 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.423 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T14:08:48.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:48.508 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone38 from 0xb521de1 (overlap 0xb39359) to 0x8fa914 2024-09-16T14:08:48.512 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:48.506+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:48.554 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.554 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:48.601 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-16T14:08:48.614 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:48.610+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:48.665 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.666 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:48.722 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:48.720+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:48.774 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.775 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:48.816 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:48.814+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:48.875 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.875 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:48.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:48.931 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:48.926+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:48.992 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:48.993 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:49.002 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:49.010 INFO:teuthology.orchestra.run.smithi079.stdout:4639 read 0x62f8f28 thru 0x6308893 (0xf96c bytes) 2024-09-16T14:08:49.018 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:49.016+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:49.024 INFO:teuthology.orchestra.run.smithi079.stdout:4641 read 0x4101ebf thru 0x410ca5c (0xab9e bytes) 2024-09-16T14:08:49.028 INFO:teuthology.orchestra.run.smithi079.stdout:4642 writesame 0x7efd122 thru 0x7f098b1 data_size 0x10(0xc790 bytes) 2024-09-16T14:08:49.112 INFO:teuthology.orchestra.run.smithi079.stdout:4644 trunc from 0xb521de1 to 0x74703e7 2024-09-16T14:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:49.119 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:49.116+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:49.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:49.159 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:49.164 INFO:teuthology.orchestra.run.smithi079.stdout:4646 trunc from 0x74703e7 to 0x584b183 2024-09-16T14:08:49.211 INFO:teuthology.orchestra.run.smithi079.stdout:4647 read 0x34bfbe3 thru 0x34c85a0 (0x89be bytes) 2024-09-16T14:08:49.217 INFO:teuthology.orchestra.run.smithi079.stdout:4650 read 0x41bbe17 thru 0x41c0c3f (0x4e29 bytes) 2024-09-16T14:08:49.221 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:49.216+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:49.223 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:49.223 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:49.232 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:49.251 INFO:teuthology.orchestra.run.smithi079.stdout:4652 write 0x62234fc thru 0x6231ea9 (0xe9ae bytes) 2024-09-16T14:08:49.257 INFO:teuthology.orchestra.run.smithi079.stdout:4653 read 0xea431 thru 0xed02e (0x2bfe bytes) 2024-09-16T14:08:49.261 INFO:teuthology.orchestra.run.smithi079.stdout:4654 write 0x535e71b thru 0x5363e01 (0x56e7 bytes) 2024-09-16T14:08:49.319 INFO:teuthology.orchestra.run.smithi079.stdout:4655 writesame 0x606c2e6 thru 0x60715a1 data_size 0x1e(0x52bc bytes) 2024-09-16T14:08:49.321 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:49.316+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:49.327 INFO:teuthology.orchestra.run.smithi079.stdout:4657 read 0x1498597 thru 0x14a7cbc (0xf726 bytes) 2024-09-16T14:08:49.330 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:49.330 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:49.339 INFO:teuthology.orchestra.run.smithi079.stdout:4658 trunc from 0x6231eaa to 0x3ed7e2a 2024-09-16T14:08:49.340 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:49.415 INFO:teuthology.orchestra.run.smithi079.stdout:4659 write 0x828ad7e thru 0x8298250 (0xd4d3 bytes) 2024-09-16T14:08:49.422 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:49.420+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:49.423 INFO:teuthology.orchestra.run.smithi079.stdout:4661 punch from 0x1760280 to 0x176e74a, (0xe4ca bytes) 2024-09-16T14:08:49.423 INFO:teuthology.orchestra.run.smithi079.stdout:4663 write 0xdbc416 thru 0xdc3472 (0x705d bytes) 2024-09-16T14:08:49.439 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:08:49.441 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:49.441 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:49.443 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T14:08:49.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:49.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:49.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T14:08:49.458 INFO:teuthology.orchestra.run.smithi079.stdout:4664 read 0x5d0b455 thru 0x5d17dbf (0xc96b bytes) 2024-09-16T14:08:49.458 INFO:teuthology.orchestra.run.smithi079.stdout:4665 read 0x7a05f6f thru 0x7a12725 (0xc7b7 bytes) 2024-09-16T14:08:49.459 INFO:teuthology.orchestra.run.smithi079.stdout:4666 read 0x12f10c8 thru 0x12f966d (0x85a6 bytes) 2024-09-16T14:08:49.499 INFO:teuthology.orchestra.run.smithi079.stdout:4667 writesame 0xaee14b8 thru 0xaeeb1ae data_size 0xd(0x9cf7 bytes) 2024-09-16T14:08:49.499 INFO:teuthology.orchestra.run.smithi079.stdout:4668 punch from 0x520ae25 to 0x520b48b, (0x666 bytes) 2024-09-16T14:08:49.506 INFO:teuthology.orchestra.run.smithi079.stdout:4669 trunc from 0xaeeb1af to 0x63b5073 2024-09-16T14:08:49.520 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:49.520 INFO:teuthology.orchestra.run.smithi079.stdout:4672 read 0x447528b thru 0x44752ad (0x23 bytes) 2024-09-16T14:08:49.520 INFO:teuthology.orchestra.run.smithi079.stdout:4674 punch from 0x5435a63 to 0x543c043, (0x65e0 bytes) 2024-09-16T14:08:49.520 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:49.523 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:49.518+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:49.526 INFO:teuthology.orchestra.run.smithi079.stdout:4680 trunc from 0x63b5073 to 0x1ce6fc0 2024-09-16T14:08:49.547 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:49.547 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:49.556 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:49.576 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:49.596 INFO:teuthology.orchestra.run.smithi079.stdout:4684 write 0x95aabbc thru 0x95aebf1 (0x4036 bytes) 2024-09-16T14:08:49.597 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:49.601 INFO:teuthology.orchestra.run.smithi079.stdout:4685 punch from 0x5c4a6eb to 0x5c58ff2, (0xe907 bytes) 2024-09-16T14:08:49.607 INFO:teuthology.orchestra.run.smithi079.stdout:4686 trunc from 0x95aebf2 to 0xcb53c5e 2024-09-16T14:08:49.613 INFO:teuthology.orchestra.run.smithi079.stdout:4687 write 0x28c2d5b thru 0x28ca093 (0x7339 bytes) 2024-09-16T14:08:49.619 INFO:teuthology.orchestra.run.smithi079.stdout:4688 punch from 0x3fd6a94 to 0x3fdf10f, (0x867b bytes) 2024-09-16T14:08:49.619 INFO:teuthology.orchestra.run.smithi079.stdout:4689 read 0x6a5665c thru 0x6a5fe43 (0x97e8 bytes) 2024-09-16T14:08:49.620 INFO:teuthology.orchestra.run.smithi079.stdout:4690 write 0xa569ebe thru 0xa56fd78 (0x5ebb bytes) 2024-09-16T14:08:49.620 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:49.624 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:49.620+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:49.626 INFO:teuthology.orchestra.run.smithi079.stdout:4691 read 0x416bbb3 thru 0x416f7e7 (0x3c35 bytes) 2024-09-16T14:08:49.628 INFO:teuthology.orchestra.run.smithi079.stdout:4693 trunc from 0xcb53c5e to 0xb47cebf 2024-09-16T14:08:49.636 INFO:teuthology.orchestra.run.smithi079.stdout:4694 read 0x32ebd79 thru 0x32f726c (0xb4f4 bytes) 2024-09-16T14:08:49.636 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:49.636 INFO:teuthology.orchestra.run.smithi079.stdout:4697 punch from 0x7e9d593 to 0x7ea0052, (0x2abf bytes) 2024-09-16T14:08:49.636 INFO:teuthology.orchestra.run.smithi079.stdout:4698 write 0x4622928 thru 0x4630baf (0xe288 bytes) 2024-09-16T14:08:49.640 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:49.643 INFO:teuthology.orchestra.run.smithi079.stdout:4700 writesame 0x615d4d5 thru 0x615eba2 data_size 0x15(0x16ce bytes) 2024-09-16T14:08:49.650 INFO:teuthology.orchestra.run.smithi079.stdout:4701 write 0x6d6d5ac thru 0x6d6e53e (0xf93 bytes) 2024-09-16T14:08:49.651 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:49.651 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:49.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:49.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:49.664 INFO:teuthology.orchestra.run.smithi079.stdout:4703 write 0xbfa0311 thru 0xbfaba79 (0xb769 bytes) 2024-09-16T14:08:49.670 INFO:teuthology.orchestra.run.smithi079.stdout:4704 write 0x207be6 thru 0x208932 (0xd4d bytes) 2024-09-16T14:08:49.693 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:49.705 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:49.706 INFO:teuthology.orchestra.run.smithi079.stdout:4706 writesame 0x75ff3f6 thru 0x760e547 data_size 0x4f(0xf152 bytes) 2024-09-16T14:08:49.716 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:49.716 INFO:teuthology.orchestra.run.smithi079.stdout:4709 read 0x1245a45 thru 0x124a910 (0x4ecc bytes) 2024-09-16T14:08:49.717 INFO:teuthology.orchestra.run.smithi079.stdout:4710 clone 40 order 18 su 32768 sc 15 2024-09-16T14:08:49.718 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:49.725 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:49.720+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:49.741 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:49.759 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:49.760 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:49.760 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:49.761 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:49.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:49.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:49.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:49.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:49.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:49.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:49.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:49.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:49.769 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:49.826 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:49.818+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:49.874 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.874 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.495700743, 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:08:49.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:08:49.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.491769325, 2024-09-16T14:08:49.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:08:49.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.491745133000002, 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:49.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.488025862000001, 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.889 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.484109097000001, 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:08:49.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:49.898 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.479726818000003, 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:49.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:08:49.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.479702977000002, 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:08:49.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.907 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.479687789000003, 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.908 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:08:49.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:49.912 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.479673202999997, 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:49.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.479569552999997, 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:49.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:08:49.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.479541826999998, 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:08:49.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:08:49.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.476868211000003, 2024-09-16T14:08:49.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.476841497000002, 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.931 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:08:49.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.476801330000001, 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:08:49.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:08:49.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.476711614999999, 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.940 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:08:49.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.476302810999996, 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:08:49.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:08:49.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.949 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.476202626999999, 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:49.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:08:49.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:08:49.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.475303988999997, 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:49.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.468105256999998, 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:49.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:08:49.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:08:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.465936964999997, 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:49.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:49.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:49.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:08:49.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:49.969 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:49.971 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:49.971 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:49.966+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:50.071 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:50.066+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:50.076 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:50.076 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:50.077 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:50.077 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:50.086 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:50.173 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:50.166+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:50.233 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:50.234 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:50.234 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:50.243 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:50.291 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:50.286+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.927898310000003, 2024-09-16T14:08:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:08:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.089935986999997, 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:08:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:08:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:08:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.562696999, 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:08:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.562673768, 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:08:50.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.562650820999998, 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:08:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.562634312, 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:08:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:08:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.562616685999998, 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:08:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:08:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.556700941999999, 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:08:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.556683537000001, 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:08:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.556329171000002, 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:08:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.553185312, 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:08:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.396 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.553170026, 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:08:50.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:08:50.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.553153617, 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.401 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:08:50.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:08:50.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.405 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.553078419999999, 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.406 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:08:50.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:08:50.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.552980872999999, 2024-09-16T14:08:50.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:08:50.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.414 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.552955632, 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.415 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:08:50.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.419 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.552926578000001, 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:08:50.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:08:50.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:50.424 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.549662589, 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:08:50.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:08:50.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.428 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.549644388000001, 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:08:50.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.549605209999999, 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:08:50.433 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:08:50.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:08:50.437 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:50.438 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:50.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:50.440 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:50.434+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:50.450 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T14:08:50.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T14:08:50.540 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:50.534+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:50.547 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:50.547 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:50.547 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:50.547 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:50.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:50.557 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone39 from 0xbfaba7a (overlap 0x1ce6fc0) to 0xe737e0 2024-09-16T14:08:50.641 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:50.636+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:50.701 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:50.701 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:50.701 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:50.701 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:50.702 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:50.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:50.743 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:50.738+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.600317624999999, 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.844 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24016458199999999, 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:08:50.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:08:50.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:08:50.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24015558300000001, 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:08:50.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:08:50.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24014723199999999, 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:08:50.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:08:50.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24011454800000001, 2024-09-16T14:08:50.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.862 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24010531099999999, 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:08:50.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:08:50.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:08:50.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24009677900000001, 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:08:50.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:08:50.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:08:50.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.240088202, 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:08:50.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24007884099999999, 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:08:50.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:08:50.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24007072900000001, 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:08:50.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.887 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24005902600000001, 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24005041299999999, 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:08:50.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24004230700000001, 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:08:50.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:08:50.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.902 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24003396499999999, 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:08:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:08:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:08:50.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24002542700000001, 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:08:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:08:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:08:50.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24001709500000001, 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.913 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:08:50.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.24000885299999999, 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:08:50.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:08:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:08:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.23999929, 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.927 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.239989909, 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:08:50.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.239980268, 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:08:50.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:08:50.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:08:50.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:50.937 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:50.937 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:50.938 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:50.939 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:50.930+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:51.035 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:51.035 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:51.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.035 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:163028 3.c 3:313fd65b:::rbd_header.113699bfa25d:head [call rbd.get_size in=19b,call rbd.get_features in=24b,call rbd.get_flags in=20b,call rbd.get_snapcontext in=18b,call lock.get_info in=30b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:51.035 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:51.034234+0000", 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00012729399999999999, 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000151822, 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 163028 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:51.034234+0000", 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:51.034234+0000", 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:51.034236+0000", 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1820000000000001e-06 2024-09-16T14:08:51.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:51.034241+0000", 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7690000000000002e-06 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:51.034259+0000", 2024-09-16T14:08:51.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7408e-05 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:51.034266+0000", 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:51.034310+0000", 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4583999999999999e-05 2024-09-16T14:08:51.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:51.034364+0000", 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3788000000000001e-05 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-16T14:08:51.040 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:51.041 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-16T14:08:51.041 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:51.034+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:51.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:51.069 INFO:teuthology.orchestra.run.smithi079.stdout:4711 read 0xae7cac8 thru 0xae86a8f (0x9fc8 bytes) 2024-09-16T14:08:51.078 INFO:teuthology.orchestra.run.smithi079.stdout:4712 write 0x77a7981 thru 0x77b625a (0xe8da bytes) 2024-09-16T14:08:51.096 INFO:teuthology.orchestra.run.smithi079.stdout:4713 write 0xb442cb8 thru 0xb45295a (0xfca3 bytes) 2024-09-16T14:08:51.104 INFO:teuthology.orchestra.run.smithi079.stdout:4714 write 0xa0e555a thru 0xa0e6cb3 (0x175a bytes) 2024-09-16T14:08:51.110 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:51.110 INFO:teuthology.orchestra.run.smithi079.stdout:4718 trunc from 0xbfaba7a to 0xc361fb5 2024-09-16T14:08:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:4720 read 0x7ade916 thru 0x7ae727d (0x8968 bytes) 2024-09-16T14:08:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:4721 punch from 0x29850a0 to 0x299168a, (0xc5ea bytes) 2024-09-16T14:08:51.121 INFO:teuthology.orchestra.run.smithi079.stdout:4722 punch from 0x40f3f0a to 0x40fb8ce, (0x79c4 bytes) 2024-09-16T14:08:51.126 INFO:teuthology.orchestra.run.smithi079.stdout:4723 trunc from 0xc361fb5 to 0xeb9a225 2024-09-16T14:08:51.136 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:51.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:51.137 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:51.137 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:51.137 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:51.140 INFO:teuthology.orchestra.run.smithi079.stdout:4724 punch from 0x1009a7b to 0x10190e0, (0xf665 bytes) 2024-09-16T14:08:51.140 INFO:teuthology.orchestra.run.smithi079.stdout:4725 write 0xb1862f3 thru 0xb188363 (0x2071 bytes) 2024-09-16T14:08:51.143 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:51.140+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:51.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:51.151 INFO:teuthology.orchestra.run.smithi079.stdout:4726 punch from 0x706f700 to 0x70779d8, (0x82d8 bytes) 2024-09-16T14:08:51.151 INFO:teuthology.orchestra.run.smithi079.stdout:4728 punch from 0x93e7fb6 to 0x93e92ca, (0x1314 bytes) 2024-09-16T14:08:51.174 INFO:teuthology.orchestra.run.smithi079.stdout:4729 trunc from 0xeb9a225 to 0x1a5a6bb 2024-09-16T14:08:51.243 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:51.240+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:51.257 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:51.257 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:51.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:51.257 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343988766000002, 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.258 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:51.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:51.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343979038999997, 2024-09-16T14:08:51.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:51.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343955475999998, 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:51.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.273 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343928237, 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.274 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:51.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:51.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343918823999999, 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:51.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343894505000002, 2024-09-16T14:08:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343874898999999, 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343854710999999, 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343831442000003, 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343801192000001, 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.34378444, 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343767176, 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343728339999998, 2024-09-16T14:08:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343693512000002, 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343657626000002, 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343635034000002, 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343605961999998, 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.347 INFO:teuthology.orchestra.run.smithi079.stdout:4730 write 0xc3ff2ea thru 0xc40779f (0x84b6 bytes) 2024-09-16T14:08:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343587616000001, 2024-09-16T14:08:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343570276999998, 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.343554353999998, 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.372 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:51.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:51.375 INFO:teuthology.orchestra.run.smithi079.stdout:4731 write 0x66a467d thru 0x66ac620 (0x7fa4 bytes) 2024-09-16T14:08:51.375 INFO:teuthology.orchestra.run.smithi079.stdout:4732 read 0x3e865e2 thru 0x3e888bf (0x22de bytes) 2024-09-16T14:08:51.375 INFO:teuthology.orchestra.run.smithi079.stdout:4733 read 0x521eb7a thru 0x5221c24 (0x30ab bytes) 2024-09-16T14:08:51.375 INFO:teuthology.orchestra.run.smithi079.stdout:4734 read 0xb33d190 thru 0xb3427ab (0x561c bytes) 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:4735 read 0x78286ce thru 0x782c497 (0x3dca bytes) 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:4736 write 0x77befdc thru 0x77c3f39 (0x4f5e bytes) 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:4737 trunc from 0xc4077a0 to 0x6fa549d 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:4738 read 0x17d864d thru 0x17e7535 (0xeee9 bytes) 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:4739 punch from 0x3de43e7 to 0x3de8d1d, (0x4936 bytes) 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:4740 read 0x5f954ba thru 0x5f9c2cf (0x6e16 bytes) 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:4741 write 0xc35b1d thru 0xc38655 (0x2b39 bytes) 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:truncating to largest ever: 0xee5aa45 2024-09-16T14:08:51.376 INFO:teuthology.orchestra.run.smithi079.stdout:4743 trunc from 0x6fa549d to 0xee5aa45 2024-09-16T14:08:51.377 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:51.370+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:51.383 INFO:teuthology.orchestra.run.smithi079.stdout:4744 trunc from 0xee5aa45 to 0xe5594d0 2024-09-16T14:08:51.401 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:51.401 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:51.401 INFO:teuthology.orchestra.run.smithi079.stdout:4747 write 0x45446c4 thru 0x4545753 (0x1090 bytes) 2024-09-16T14:08:51.405 INFO:teuthology.orchestra.run.smithi079.stdout:4748 writesame 0x708bd4a thru 0x708e27d data_size 0x2(0x2534 bytes) 2024-09-16T14:08:51.413 INFO:teuthology.orchestra.run.smithi079.stdout:4751 writesame 0xc8806a2 thru 0xc88a8e6 data_size 0xb7(0xa245 bytes) 2024-09-16T14:08:51.436 INFO:teuthology.orchestra.run.smithi079.stdout:4752 write 0xee56933 thru 0xee60cf0 (0xa3be bytes) 2024-09-16T14:08:51.453 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T14:08:51.454 INFO:teuthology.orchestra.run.smithi079.stdout:4753 write 0xf541d9 thru 0xf5795c (0x3784 bytes) 2024-09-16T14:08:51.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:08:51.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T14:08:51.466 INFO:teuthology.orchestra.run.smithi079.stdout:4754 read 0x56d4e16 thru 0x56df9aa (0xab95 bytes) 2024-09-16T14:08:51.466 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:51.466 INFO:teuthology.orchestra.run.smithi079.stdout:4757 write 0x65ec0bf thru 0x65f5171 (0x90b3 bytes) 2024-09-16T14:08:51.475 INFO:teuthology.orchestra.run.smithi079.stdout:4758 read 0x2ff5c4b thru 0x2ffca78 (0x6e2e bytes) 2024-09-16T14:08:51.475 INFO:teuthology.orchestra.run.smithi079.stdout:4759 clone 41 order 24 su 262144 sc 12 2024-09-16T14:08:51.476 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:51.472+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:51.488 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:51.488 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:51.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:51.577 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:51.570+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:51.641 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:51.641 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:51.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:51.678 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:51.672+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:51.743 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.836506104999998, 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:08:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:08:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.836483397000002, 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:08:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:08:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.836459245, 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.836426304, 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:08:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.836401621, 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:08:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:08:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.836378392, 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:08:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.834532428000003, 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:08:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.834489564000002, 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:08:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:08:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.833029946000003, 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:08:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.833014177000003, 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:08:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832998425, 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:08:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:08:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832982993999998, 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:08:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:08:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832967355999997, 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:08:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832952216999999, 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:08:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:08:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:08:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832937379999997, 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832914375999998, 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:08:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832899150999999, 2024-09-16T14:08:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:08:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832883967000001, 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:08:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832868722000001, 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:08:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:08:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 57.832853563999997, 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:08:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:08:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:08:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:08:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:51.848 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:51.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:51.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:51.850 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:51.846+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:51.950 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:51.946+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:52.001 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:52.001 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:52.001 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:52.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:52.030 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:52.051 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:52.048+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:52.063 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:52.084 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:52.105 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:52.120 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:52.120 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:52.127 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:52.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:52.148 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:52.172 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:52.173 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:52.168+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:52.208 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:52.210 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:52.210 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:52.210 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:52.210 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:52.210 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:52.210 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:52.211 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:52.211 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:52.259 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:52.260 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:52.260 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:52.256+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:52.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:52.354 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:52.352+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:52.376 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:52.377 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:52.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:52.455 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:52.452+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:52.467 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:08:52.468 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T14:08:52.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T14:08:52.487 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:52.487 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:52.497 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:52.555 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:52.552+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:52.591 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:52.591 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:52.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:52.609 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone40 from 0xee60cf1 (overlap 0x1a5a6bb) to 0x151522f 2024-09-16T14:08:52.656 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:52.650+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:52.697 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:52.697 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:52.706 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:52.757 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:52.750+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:52.810 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:52.810 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:52.819 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:52.858 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:52.850+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:52.912 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:52.912 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:52.921 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:52.942 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-16T14:08:52.959 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:52.956+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:53.059 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:53.056+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:53.063 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:53.064 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:53.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:53.160 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:53.156+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:53.176 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:53.176 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:53.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:53.262 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:53.260+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:53.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:53.362 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:53.360+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:53.397 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:53.398 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:53.408 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:53.464 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:53.458+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:53.477 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T14:08:53.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T14:08:53.508 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:53.508 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:53.518 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:53.533 INFO:teuthology.orchestra.run.smithi079.stdout:4760 write 0x7e99009 thru 0x7e9ed39 (0x5d31 bytes) 2024-09-16T14:08:53.565 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:53.560+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:53.612 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:53.612 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:53.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:53.649 INFO:teuthology.orchestra.run.smithi079.stdout:4761 read 0x753a718 thru 0x7545942 (0xb22b bytes) 2024-09-16T14:08:53.661 INFO:teuthology.orchestra.run.smithi079.stdout:4762 trunc from 0xee60cf1 to 0x8dba5d2 2024-09-16T14:08:53.678 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:53.676+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:53.733 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:53.733 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:53.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:53.767 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:53.762+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:53.869 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:53.862+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:53.931 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:53.931 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:53.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:53.969 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:53.964+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:54.070 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:54.066+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:54.079 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:54.079 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:54.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:54.172 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:54.168+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:54.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:54.261 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:54.261 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:54.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:54.297 INFO:teuthology.orchestra.run.smithi079.stdout:4764 read 0x3057f16 thru 0x3063e5a (0xbf45 bytes) 2024-09-16T14:08:54.298 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:54.292+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:54.299 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.299 INFO:teuthology.orchestra.run.smithi079.stdout:4767 read 0x9bb697 thru 0x9cabd4 (0xf53e bytes) 2024-09-16T14:08:54.300 INFO:teuthology.orchestra.run.smithi079.stdout:4768 writesame 0x845f145 thru 0x8463809 data_size 0x3d(0x46c5 bytes) 2024-09-16T14:08:54.304 INFO:teuthology.orchestra.run.smithi079.stdout:4769 read 0x46016e2 thru 0x460f003 (0xd922 bytes) 2024-09-16T14:08:54.305 INFO:teuthology.orchestra.run.smithi079.stdout:4770 write 0x4b81f5b thru 0x4b8589a (0x3940 bytes) 2024-09-16T14:08:54.312 INFO:teuthology.orchestra.run.smithi079.stdout:4771 trunc from 0x8dba5d2 to 0x6e0633f 2024-09-16T14:08:54.326 INFO:teuthology.orchestra.run.smithi079.stdout:4772 writesame 0x1ace563 thru 0x1aceca3 data_size 0x1(0x741 bytes) 2024-09-16T14:08:54.329 INFO:teuthology.orchestra.run.smithi079.stdout:4773 write 0x1f74af2 thru 0x1f8358a (0xea99 bytes) 2024-09-16T14:08:54.332 INFO:teuthology.orchestra.run.smithi079.stdout:4774 read 0x41658b4 thru 0x41659bb (0x108 bytes) 2024-09-16T14:08:54.339 INFO:teuthology.orchestra.run.smithi079.stdout:4775 trunc from 0x6e0633f to 0x38303de 2024-09-16T14:08:54.353 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.362 INFO:teuthology.orchestra.run.smithi079.stdout:4777 write 0x49e6251 thru 0x49f0e0c (0xabbc bytes) 2024-09-16T14:08:54.365 INFO:teuthology.orchestra.run.smithi079.stdout:4778 read 0x2aace68 thru 0x2abba1b (0xebb4 bytes) 2024-09-16T14:08:54.366 INFO:teuthology.orchestra.run.smithi079.stdout:4780 punch from 0x28d0d2a to 0x28dc10b, (0xb3e1 bytes) 2024-09-16T14:08:54.368 INFO:teuthology.orchestra.run.smithi079.stdout:4781 trunc from 0x49f0e0d to 0x5f4194a 2024-09-16T14:08:54.372 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:54.373 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:54.366+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:54.378 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.379 INFO:teuthology.orchestra.run.smithi079.stdout:4783 read 0x2c356d6 thru 0x2c35ed8 (0x803 bytes) 2024-09-16T14:08:54.394 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:54.396 INFO:teuthology.orchestra.run.smithi079.stdout:4784 writesame 0x8e8a10c thru 0x8e95ca0 data_size 0x1(0xbb95 bytes) 2024-09-16T14:08:54.398 INFO:teuthology.orchestra.run.smithi079.stdout:4785 punch from 0x1032117 to 0x1034994, (0x287d bytes) 2024-09-16T14:08:54.398 INFO:teuthology.orchestra.run.smithi079.stdout:4786 read 0x55b0364 thru 0x55bd9cd (0xd66a bytes) 2024-09-16T14:08:54.398 INFO:teuthology.orchestra.run.smithi079.stdout:4788 read 0x2f69451 thru 0x2f6bdd6 (0x2986 bytes) 2024-09-16T14:08:54.399 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.399 INFO:teuthology.orchestra.run.smithi079.stdout:4792 punch from 0x39a9d1d to 0x39b5cab, (0xbf8e bytes) 2024-09-16T14:08:54.399 INFO:teuthology.orchestra.run.smithi079.stdout:4794 read 0x65fdf39 thru 0x6600dec (0x2eb4 bytes) 2024-09-16T14:08:54.399 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.399 INFO:teuthology.orchestra.run.smithi079.stdout:4798 read 0x8c028f5 thru 0x8c0b52e (0x8c3a bytes) 2024-09-16T14:08:54.400 INFO:teuthology.orchestra.run.smithi079.stdout:4800 read 0x63fa0a4 thru 0x6406a6f (0xc9cc bytes) 2024-09-16T14:08:54.401 INFO:teuthology.orchestra.run.smithi079.stdout:4801 read 0x58071f7 thru 0x5812466 (0xb270 bytes) 2024-09-16T14:08:54.401 INFO:teuthology.orchestra.run.smithi079.stdout:4804 writesame 0x82684fe thru 0x826c21d data_size 0x30(0x3d20 bytes) 2024-09-16T14:08:54.404 INFO:teuthology.orchestra.run.smithi079.stdout:4805 write 0x68bdb20 thru 0x68ca96b (0xce4c bytes) 2024-09-16T14:08:54.406 INFO:teuthology.orchestra.run.smithi079.stdout:4806 write 0x59b2b78 thru 0x59c04b8 (0xd941 bytes) 2024-09-16T14:08:54.410 INFO:teuthology.orchestra.run.smithi079.stdout:4807 read 0x4161298 thru 0x416941d (0x8186 bytes) 2024-09-16T14:08:54.411 INFO:teuthology.orchestra.run.smithi079.stdout:4808 write 0x489416c thru 0x489f8f1 (0xb786 bytes) 2024-09-16T14:08:54.413 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:54.413 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:54.414 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:54.414 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:54.414 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:54.419 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:54.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:54.424 INFO:teuthology.orchestra.run.smithi079.stdout:4809 writesame 0x96ba4c4 thru 0x96be7cf data_size 0xe(0x430c bytes) 2024-09-16T14:08:54.427 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.427 INFO:teuthology.orchestra.run.smithi079.stdout:4811 read 0x37bd8ec thru 0x37c3dfc (0x6511 bytes) 2024-09-16T14:08:54.442 INFO:teuthology.orchestra.run.smithi079.stdout:4813 write 0xc4c6d18 thru 0xc4d1383 (0xa66c bytes) 2024-09-16T14:08:54.447 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:54.449 INFO:teuthology.orchestra.run.smithi079.stdout:4814 write 0x1eedb43 thru 0x1efd903 (0xfdc1 bytes) 2024-09-16T14:08:54.452 INFO:teuthology.orchestra.run.smithi079.stdout:4815 writesame 0x16256d8 thru 0x1625cd6 data_size 0x1(0x5ff bytes) 2024-09-16T14:08:54.471 INFO:teuthology.orchestra.run.smithi079.stdout:4816 trunc from 0xc4d1384 to 0x3bfda19 2024-09-16T14:08:54.474 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:54.470+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:54.482 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:54.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:08:54.487 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T14:08:54.501 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T14:08:54.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:08:54.507 INFO:teuthology.orchestra.run.smithi079.stdout:4818 trunc from 0x3bfda19 to 0x9ccbb1 2024-09-16T14:08:54.521 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:54.522 INFO:teuthology.orchestra.run.smithi079.stdout:4819 read 0x3cf4c4 thru 0x3d6d0c (0x7849 bytes) 2024-09-16T14:08:54.523 INFO:teuthology.orchestra.run.smithi079.stdout:4821 read 0xba418 thru 0xc5a4d (0xb636 bytes) 2024-09-16T14:08:54.524 INFO:teuthology.orchestra.run.smithi079.stdout:4823 read 0x1bb1ad thru 0x1c4f54 (0x9da8 bytes) 2024-09-16T14:08:54.546 INFO:teuthology.orchestra.run.smithi079.stdout:4824 write 0xa264991 thru 0xa273477 (0xeae7 bytes) 2024-09-16T14:08:54.550 INFO:teuthology.orchestra.run.smithi079.stdout:4825 punch from 0x2a536ed to 0x2a58272, (0x4b85 bytes) 2024-09-16T14:08:54.550 INFO:teuthology.orchestra.run.smithi079.stdout:4827 write 0x671d0dc thru 0x6724d99 (0x7cbe bytes) 2024-09-16T14:08:54.553 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.553 INFO:teuthology.orchestra.run.smithi079.stdout:4829 read 0x18d04a9 thru 0x18d240f (0x1f67 bytes) 2024-09-16T14:08:54.557 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:54.561 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:54.561 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:54.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648405722, 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:08:54.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:08:54.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648395995000001, 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:08:54.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648372432000002, 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:08:54.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:08:54.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:08:54.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648345192999997, 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:08:54.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:08:54.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648335779999996, 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:08:54.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648311460999999, 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:08:54.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648291854999997, 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:08:54.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:08:54.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:08:54.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:08:54.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:08:54.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:08:54.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:54.601 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648271667000003, 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:08:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:08:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:08:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:08:54.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648248398, 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:08:54.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:08:54.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648218147999998, 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:08:54.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:08:54.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:08:54.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:54.618 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648201395999997, 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:54.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648184131999997, 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:54.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.629 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648145296000003, 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:08:54.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:08:54.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648110467999999, 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:08:54.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.640 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648074582, 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:08:54.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648051989999999, 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.646 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:54.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648022918000002, 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.651 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:08:54.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.656 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.648004571999998, 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:08:54.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.647987233000002, 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:08:54.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:08:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.647971310000003, 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:08:54.667 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:08:54.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:54.672 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:54.673 INFO:teuthology.orchestra.run.smithi079.stdout:4831 trunc from 0xa273478 to 0xebb0ba6 2024-09-16T14:08:54.674 INFO:teuthology.orchestra.run.smithi079.stdout:4832 write 0x92dd0a thru 0x92de91 (0x188 bytes) 2024-09-16T14:08:54.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:54.675 INFO:teuthology.orchestra.run.smithi079.stdout:4833 read 0x63de260 thru 0x63e1d3d (0x3ade bytes) 2024-09-16T14:08:54.675 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:54.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:54.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:54.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:54.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:54.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:54.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:54.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:54.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4834 trunc from 0xebb0ba6 to 0x11d8b46 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4837 punch from 0x90240a to 0x90f999, (0xd58f bytes) 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4839 read 0x7afe8 thru 0x842fc (0x9315 bytes) 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4840 write 0x39501d7 thru 0x3959837 (0x9661 bytes) 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4842 writesame 0x1951b0b thru 0x1952200 data_size 0x1b(0x6f6 bytes) 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4843 read 0x2df25fa thru 0x2dfca2f (0xa436 bytes) 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4844 writesame 0x4017877 thru 0x4026265 data_size 0x1(0xe9ef bytes) 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4845 read 0x25614e6 thru 0x256e675 (0xd190 bytes) 2024-09-16T14:08:54.678 INFO:teuthology.orchestra.run.smithi079.stdout:4846 read 0x1d13b07 thru 0x1d17f71 (0x446b bytes) 2024-09-16T14:08:54.679 INFO:teuthology.orchestra.run.smithi079.stdout:4847 write 0x604fe36 thru 0x60598ab (0x9a76 bytes) 2024-09-16T14:08:54.679 INFO:teuthology.orchestra.run.smithi079.stdout:4848 write 0x52d3366 thru 0x52de1c8 (0xae63 bytes) 2024-09-16T14:08:54.679 INFO:teuthology.orchestra.run.smithi079.stdout:4849 trunc from 0x60598ac to 0xa33ddd4 2024-09-16T14:08:54.679 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:54.666+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:54.683 INFO:teuthology.orchestra.run.smithi079.stdout:4850 write 0x97488be thru 0x974bcc4 (0x3407 bytes) 2024-09-16T14:08:54.686 INFO:teuthology.orchestra.run.smithi079.stdout:4851 read 0x72daf45 thru 0x72e4f26 (0x9fe2 bytes) 2024-09-16T14:08:54.687 INFO:teuthology.orchestra.run.smithi079.stdout:4852 write 0x7d5b9d8 thru 0x7d625e8 (0x6c11 bytes) 2024-09-16T14:08:54.691 INFO:teuthology.orchestra.run.smithi079.stdout:4853 read 0x42c83dc thru 0x42d01ff (0x7e24 bytes) 2024-09-16T14:08:54.691 INFO:teuthology.orchestra.run.smithi079.stdout:4854 read 0x8d4e9f9 thru 0x8d5b2b7 (0xc8bf bytes) 2024-09-16T14:08:54.692 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:08:54.705 INFO:teuthology.orchestra.run.smithi079.stdout:4856 write 0xc0b6de6 thru 0xc0bd7c0 (0x69db bytes) 2024-09-16T14:08:54.722 INFO:teuthology.orchestra.run.smithi079.stdout:4857 trunc from 0xc0bd7c1 to 0x495867d 2024-09-16T14:08:54.754 INFO:teuthology.orchestra.run.smithi079.stdout:4858 trunc from 0x495867d to 0x7f30763 2024-09-16T14:08:54.759 INFO:teuthology.orchestra.run.smithi079.stdout:4860 write 0x16ae654 thru 0x16b302e (0x49db bytes) 2024-09-16T14:08:54.761 INFO:teuthology.orchestra.run.smithi079.stdout:4861 clone 42 order 22 su 2097152 sc 4 2024-09-16T14:08:54.774 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:54.772+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:54.790 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:54.790 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:54.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:08:54.829 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:08:54.841 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-16T14:08:54.797524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1211774754,"num_objects":2939,"num_object_clones":636,"num_object_copies":5878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2939,"num_whiteouts":735,"num_read":222535,"num_read_kb":752268,"num_write":23376,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1309,"num_bytes_recovered":89016121,"num_keys_recovered":147,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18081,"ondisk_log_size":18081,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2172884,"kb_used_data":64696,"kb_used_omap":41,"kb_used_meta":2108054,"kb_avail":658429996,"statfs":{"total":676457349120,"available":674232315904,"internally_reserved":0,"allocated":66248704,"data_stored":544753374,"data_compressed":14052404,"data_compressed_allocated":42786816,"data_compressed_original":532154934,"omap_allocated":42088,"internal_metadata":2158648216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54465830,"num_objects":117,"num_object_clones":30,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":46,"num_read":9939,"num_read_kb":30476,"num_write":658,"num_write_kb":4693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000697"},"pg_stats":[{"pgid":"3.1f","version":"103'1637","reported_seq":6463,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.412759+0000","last_change":"2024-09-16T14:07:11.338271+0000","last_active":"2024-09-16T14:08:51.412759+0000","last_peered":"2024-09-16T14:08:51.412759+0000","last_clean":"2024-09-16T14:08:51.412759+0000","last_became_active":"2024-09-16T14:07:09.540900+0000","last_became_peered":"2024-09-16T14:07:09.540900+0000","last_unstale":"2024-09-16T14:08:51.412759+0000","last_undegraded":"2024-09-16T14:08:51.412759+0000","last_fullsized":"2024-09-16T14:08:51.412759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:59.820596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233770000000001,"stat_sum":{"num_bytes":22853786,"num_objects":83,"num_object_clones":15,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":15,"num_read":14341,"num_read_kb":14697,"num_write":2790,"num_write_kb":6574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"104'589","reported_seq":2048,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.446724+0000","last_change":"2024-09-16T14:07:11.337654+0000","last_active":"2024-09-16T14:08:51.446724+0000","last_peered":"2024-09-16T14:08:51.446724+0000","last_clean":"2024-09-16T14:08:51.446724+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:08:51.446724+0000","last_undegraded":"2024-09-16T14:08:51.446724+0000","last_fullsized":"2024-09-16T14:08:51.446724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060911299999999999,"stat_sum":{"num_bytes":30496472,"num_objects":92,"num_object_clones":19,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":18,"num_read":3161,"num_read_kb":19744,"num_write":820,"num_write_kb":9189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"103'380","reported_seq":2827,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.970141+0000","last_change":"2024-09-16T14:08:28.327780+0000","last_active":"2024-09-16T14:08:50.970141+0000","last_peered":"2024-09-16T14:08:50.970141+0000","last_clean":"2024-09-16T14:08:50.970141+0000","last_became_active":"2024-09-16T14:08:28.215309+0000","last_became_peered":"2024-09-16T14:08:28.215309+0000","last_unstale":"2024-09-16T14:08:50.970141+0000","last_undegraded":"2024-09-16T14:08:50.970141+0000","last_fullsized":"2024-09-16T14:08:50.970141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:42.429361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072576699999999999,"stat_sum":{"num_bytes":35735733,"num_objects":100,"num_object_clones":21,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":21,"num_read":6078,"num_read_kb":21111,"num_write":553,"num_write_kb":12718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":5718086,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"103'719","reported_seq":13144,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.454612+0000","last_change":"2024-09-16T14:07:11.340416+0000","last_active":"2024-09-16T14:08:51.454612+0000","last_peered":"2024-09-16T14:08:51.454612+0000","last_clean":"2024-09-16T14:08:51.454612+0000","last_became_active":"2024-09-16T14:07:09.551657+0000","last_became_peered":"2024-09-16T14:07:09.551657+0000","last_unstale":"2024-09-16T14:08:51.454612+0000","last_undegraded":"2024-09-16T14:08:51.454612+0000","last_fullsized":"2024-09-16T14:08:51.454612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:49.635426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064309800000000004,"stat_sum":{"num_bytes":38411660,"num_objects":88,"num_object_clones":23,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":27,"num_read":14608,"num_read_kb":24332,"num_write":1009,"num_write_kb":8382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"103'430","reported_seq":4433,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.455995+0000","last_change":"2024-09-16T14:07:22.350217+0000","last_active":"2024-09-16T14:08:51.455995+0000","last_peered":"2024-09-16T14:08:51.455995+0000","last_clean":"2024-09-16T14:08:51.455995+0000","last_became_active":"2024-09-16T14:07:22.349546+0000","last_became_peered":"2024-09-16T14:07:22.349546+0000","last_unstale":"2024-09-16T14:08:51.455995+0000","last_undegraded":"2024-09-16T14:08:51.455995+0000","last_fullsized":"2024-09-16T14:08:51.455995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:30.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021275310000000002,"stat_sum":{"num_bytes":23725811,"num_objects":91,"num_object_clones":16,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":23,"num_read":6258,"num_read_kb":15357,"num_write":486,"num_write_kb":6241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"103'432","reported_seq":4953,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.454504+0000","last_change":"2024-09-16T14:07:11.340816+0000","last_active":"2024-09-16T14:08:51.454504+0000","last_peered":"2024-09-16T14:08:51.454504+0000","last_clean":"2024-09-16T14:08:51.454504+0000","last_became_active":"2024-09-16T14:07:09.540237+0000","last_became_peered":"2024-09-16T14:07:09.540237+0000","last_unstale":"2024-09-16T14:08:51.454504+0000","last_undegraded":"2024-09-16T14:08:51.454504+0000","last_fullsized":"2024-09-16T14:08:51.454504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:38.147579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099540499999999999,"stat_sum":{"num_bytes":22383318,"num_objects":85,"num_object_clones":20,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":23,"num_read":5359,"num_read_kb":23938,"num_write":444,"num_write_kb":21969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"104'407","reported_seq":1118,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.441574+0000","last_change":"2024-09-16T14:07:22.346713+0000","last_active":"2024-09-16T14:08:51.441574+0000","last_peered":"2024-09-16T14:08:51.441574+0000","last_clean":"2024-09-16T14:08:51.441574+0000","last_became_active":"2024-09-16T14:07:22.346031+0000","last_became_peered":"2024-09-16T14:07:22.346031+0000","last_unstale":"2024-09-16T14:08:51.441574+0000","last_undegraded":"2024-09-16T14:08:51.441574+0000","last_fullsized":"2024-09-16T14:08:51.441574+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:39.265638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003350142,"stat_sum":{"num_bytes":31595139,"num_objects":84,"num_object_clones":14,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":16,"num_read":1341,"num_read_kb":21935,"num_write":502,"num_write_kb":17534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"104'474","reported_seq":5207,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.469005+0000","last_change":"2024-09-16T14:07:11.338172+0000","last_active":"2024-09-16T14:08:51.469005+0000","last_peered":"2024-09-16T14:08:51.469005+0000","last_clean":"2024-09-16T14:08:51.469005+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:08:51.469005+0000","last_undegraded":"2024-09-16T14:08:51.469005+0000","last_fullsized":"2024-09-16T14:08:51.469005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148865,"stat_sum":{"num_bytes":37104369,"num_objects":71,"num_object_clones":12,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":15,"num_read":6916,"num_read_kb":19923,"num_write":525,"num_write_kb":7568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"103'1147","reported_seq":4513,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.182998+0000","last_change":"2024-09-16T14:08:31.367987+0000","last_active":"2024-09-16T14:08:51.182998+0000","last_peered":"2024-09-16T14:08:51.182998+0000","last_clean":"2024-09-16T14:08:51.182998+0000","last_became_active":"2024-09-16T14:08:31.364152+0000","last_became_peered":"2024-09-16T14:08:31.364152+0000","last_unstale":"2024-09-16T14:08:51.182998+0000","last_undegraded":"2024-09-16T14:08:51.182998+0000","last_fullsized":"2024-09-16T14:08:51.182998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:05.530637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215788,"stat_sum":{"num_bytes":31074832,"num_objects":89,"num_object_clones":21,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":26,"num_read":8110,"num_read_kb":32761,"num_write":1847,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1214676,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"102'453","reported_seq":6626,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.459065+0000","last_change":"2024-09-16T14:08:32.368069+0000","last_active":"2024-09-16T14:08:51.459065+0000","last_peered":"2024-09-16T14:08:51.459065+0000","last_clean":"2024-09-16T14:08:51.459065+0000","last_became_active":"2024-09-16T14:08:32.365885+0000","last_became_peered":"2024-09-16T14:08:32.365885+0000","last_unstale":"2024-09-16T14:08:51.459065+0000","last_undegraded":"2024-09-16T14:08:51.459065+0000","last_fullsized":"2024-09-16T14:08:51.459065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:53.426423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500454,"stat_sum":{"num_bytes":45180005,"num_objects":105,"num_object_clones":23,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":27,"num_read":7555,"num_read_kb":25286,"num_write":477,"num_write_kb":9154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":3589265,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"104'310","reported_seq":3954,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.450858+0000","last_change":"2024-09-16T14:07:11.343447+0000","last_active":"2024-09-16T14:08:51.450858+0000","last_peered":"2024-09-16T14:08:51.450858+0000","last_clean":"2024-09-16T14:08:51.450858+0000","last_became_active":"2024-09-16T14:07:09.549545+0000","last_became_peered":"2024-09-16T14:07:09.549545+0000","last_unstale":"2024-09-16T14:08:51.450858+0000","last_undegraded":"2024-09-16T14:08:51.450858+0000","last_fullsized":"2024-09-16T14:08:51.450858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:27.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012153120000000001,"stat_sum":{"num_bytes":35475058,"num_objects":96,"num_object_clones":29,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":30,"num_read":3686,"num_read_kb":23719,"num_write":332,"num_write_kb":6356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"103'472","reported_seq":3871,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.196472+0000","last_change":"2024-09-16T14:08:30.361920+0000","last_active":"2024-09-16T14:08:51.196472+0000","last_peered":"2024-09-16T14:08:51.196472+0000","last_clean":"2024-09-16T14:08:51.196472+0000","last_became_active":"2024-09-16T14:08:30.360514+0000","last_became_peered":"2024-09-16T14:08:30.360514+0000","last_unstale":"2024-09-16T14:08:51.196472+0000","last_undegraded":"2024-09-16T14:08:51.196472+0000","last_fullsized":"2024-09-16T14:08:51.196472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:15.505375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009984429999999999,"stat_sum":{"num_bytes":44661799,"num_objects":88,"num_object_clones":18,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":19,"num_read":5437,"num_read_kb":25061,"num_write":505,"num_write_kb":7261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":1548718,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444450+0000","last_change":"2024-09-16T14:08:28.215419+0000","last_active":"2024-09-16T14:08:49.444450+0000","last_peered":"2024-09-16T14:08:49.444450+0000","last_clean":"2024-09-16T14:08:49.444450+0000","last_became_active":"2024-09-16T14:08:28.215112+0000","last_became_peered":"2024-09-16T14:08:28.215112+0000","last_unstale":"2024-09-16T14:08:49.444450+0000","last_undegraded":"2024-09-16T14:08:49.444450+0000","last_fullsized":"2024-09-16T14:08:49.444450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:19.841089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"103'565","reported_seq":3827,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.575054+0000","last_change":"2024-09-16T14:08:41.366677+0000","last_active":"2024-09-16T14:08:50.575054+0000","last_peered":"2024-09-16T14:08:50.575054+0000","last_clean":"2024-09-16T14:08:50.575054+0000","last_became_active":"2024-09-16T14:08:41.361496+0000","last_became_peered":"2024-09-16T14:08:41.361496+0000","last_unstale":"2024-09-16T14:08:50.575054+0000","last_undegraded":"2024-09-16T14:08:50.575054+0000","last_fullsized":"2024-09-16T14:08:50.575054+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:12.420908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032490359999999999,"stat_sum":{"num_bytes":43515179,"num_objects":95,"num_object_clones":19,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":16,"num_read":4633,"num_read_kb":21891,"num_write":704,"num_write_kb":8605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":3405893,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.496070+0000","last_change":"2024-09-16T14:07:22.350069+0000","last_active":"2024-09-16T14:08:50.496070+0000","last_peered":"2024-09-16T14:08:50.496070+0000","last_clean":"2024-09-16T14:08:50.496070+0000","last_became_active":"2024-09-16T14:07:22.349757+0000","last_became_peered":"2024-09-16T14:07:22.349757+0000","last_unstale":"2024-09-16T14:08:50.496070+0000","last_undegraded":"2024-09-16T14:08:50.496070+0000","last_fullsized":"2024-09-16T14:08:50.496070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:57.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"103'741","reported_seq":6400,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.304500+0000","last_change":"2024-09-16T14:07:22.351951+0000","last_active":"2024-09-16T14:08:51.304500+0000","last_peered":"2024-09-16T14:08:51.304500+0000","last_clean":"2024-09-16T14:08:51.304500+0000","last_became_active":"2024-09-16T14:07:22.351394+0000","last_became_peered":"2024-09-16T14:07:22.351394+0000","last_unstale":"2024-09-16T14:08:51.304500+0000","last_undegraded":"2024-09-16T14:08:51.304500+0000","last_fullsized":"2024-09-16T14:08:51.304500+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:55.945287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027484229999999998,"stat_sum":{"num_bytes":76394086,"num_objects":93,"num_object_clones":20,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":26,"num_read":9079,"num_read_kb":41824,"num_write":1091,"num_write_kb":28785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444443+0000","last_change":"2024-09-16T14:08:28.213074+0000","last_active":"2024-09-16T14:08:49.444443+0000","last_peered":"2024-09-16T14:08:49.444443+0000","last_clean":"2024-09-16T14:08:49.444443+0000","last_became_active":"2024-09-16T14:08:28.212640+0000","last_became_peered":"2024-09-16T14:08:28.212640+0000","last_unstale":"2024-09-16T14:08:49.444443+0000","last_undegraded":"2024-09-16T14:08:49.444443+0000","last_fullsized":"2024-09-16T14:08:49.444443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:09.532319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"103'393","reported_seq":3103,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.344959+0000","last_change":"2024-09-16T14:07:11.547417+0000","last_active":"2024-09-16T14:08:51.344959+0000","last_peered":"2024-09-16T14:08:51.344959+0000","last_clean":"2024-09-16T14:08:51.344959+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:08:51.344959+0000","last_undegraded":"2024-09-16T14:08:51.344959+0000","last_fullsized":"2024-09-16T14:08:51.344959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097608199999999997,"stat_sum":{"num_bytes":15659686,"num_objects":88,"num_object_clones":20,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":26,"num_read":3960,"num_read_kb":12390,"num_write":424,"num_write_kb":5434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.442372+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:08:50.442372+0000","last_peered":"2024-09-16T14:08:50.442372+0000","last_clean":"2024-09-16T14:08:50.442372+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:08:50.442372+0000","last_undegraded":"2024-09-16T14:08:50.442372+0000","last_fullsized":"2024-09-16T14:08:50.442372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"103'544","reported_seq":8558,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.207546+0000","last_change":"2024-09-16T14:07:11.361331+0000","last_active":"2024-09-16T14:08:51.207546+0000","last_peered":"2024-09-16T14:08:51.207546+0000","last_clean":"2024-09-16T14:08:51.207546+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:08:51.207546+0000","last_undegraded":"2024-09-16T14:08:51.207546+0000","last_fullsized":"2024-09-16T14:08:51.207546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062220600000000004,"stat_sum":{"num_bytes":26234909,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":23,"num_read":9302,"num_read_kb":11175,"num_write":684,"num_write_kb":4642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"103'896","reported_seq":3672,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.203934+0000","last_change":"2024-09-16T14:07:11.342816+0000","last_active":"2024-09-16T14:08:51.203934+0000","last_peered":"2024-09-16T14:08:51.203934+0000","last_clean":"2024-09-16T14:08:51.203934+0000","last_became_active":"2024-09-16T14:07:09.549842+0000","last_became_peered":"2024-09-16T14:07:09.549842+0000","last_unstale":"2024-09-16T14:08:51.203934+0000","last_undegraded":"2024-09-16T14:08:51.203934+0000","last_fullsized":"2024-09-16T14:08:51.203934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:47.724999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014467779999999999,"stat_sum":{"num_bytes":37384569,"num_objects":95,"num_object_clones":18,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":7877,"num_read_kb":16086,"num_write":1175,"num_write_kb":9291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":53,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444325+0000","last_change":"2024-09-16T14:08:28.217645+0000","last_active":"2024-09-16T14:08:49.444325+0000","last_peered":"2024-09-16T14:08:49.444325+0000","last_clean":"2024-09-16T14:08:49.444325+0000","last_became_active":"2024-09-16T14:08:28.217418+0000","last_became_peered":"2024-09-16T14:08:28.217418+0000","last_unstale":"2024-09-16T14:08:49.444325+0000","last_undegraded":"2024-09-16T14:08:49.444325+0000","last_fullsized":"2024-09-16T14:08:49.444325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:25.918964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"103'1008","reported_seq":4527,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.475021+0000","last_change":"2024-09-16T14:08:41.367878+0000","last_active":"2024-09-16T14:08:51.475021+0000","last_peered":"2024-09-16T14:08:51.475021+0000","last_clean":"2024-09-16T14:08:51.475021+0000","last_became_active":"2024-09-16T14:08:41.365767+0000","last_became_peered":"2024-09-16T14:08:41.365767+0000","last_unstale":"2024-09-16T14:08:51.475021+0000","last_undegraded":"2024-09-16T14:08:51.475021+0000","last_fullsized":"2024-09-16T14:08:51.475021+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:19:25.019511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076943300000000003,"stat_sum":{"num_bytes":51051893,"num_objects":112,"num_object_clones":26,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":31,"num_read":6811,"num_read_kb":21948,"num_write":1508,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":7223267,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.442093+0000","last_change":"2024-09-16T14:08:38.421227+0000","last_active":"2024-09-16T14:08:50.442093+0000","last_peered":"2024-09-16T14:08:50.442093+0000","last_clean":"2024-09-16T14:08:50.442093+0000","last_became_active":"2024-09-16T14:08:38.420867+0000","last_became_peered":"2024-09-16T14:08:38.420867+0000","last_unstale":"2024-09-16T14:08:50.442093+0000","last_undegraded":"2024-09-16T14:08:50.442093+0000","last_fullsized":"2024-09-16T14:08:50.442093+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:52.563336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"104'602","reported_seq":6178,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.447233+0000","last_change":"2024-09-16T14:07:11.547270+0000","last_active":"2024-09-16T14:08:51.447233+0000","last_peered":"2024-09-16T14:08:51.447233+0000","last_clean":"2024-09-16T14:08:51.447233+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:08:51.447233+0000","last_undegraded":"2024-09-16T14:08:51.447233+0000","last_fullsized":"2024-09-16T14:08:51.447233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085373199999999995,"stat_sum":{"num_bytes":34814460,"num_objects":89,"num_object_clones":17,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":24,"num_read":7567,"num_read_kb":19640,"num_write":791,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444239+0000","last_change":"2024-09-16T14:08:28.214757+0000","last_active":"2024-09-16T14:08:49.444239+0000","last_peered":"2024-09-16T14:08:49.444239+0000","last_clean":"2024-09-16T14:08:49.444239+0000","last_became_active":"2024-09-16T14:08:28.213488+0000","last_became_peered":"2024-09-16T14:08:28.213488+0000","last_unstale":"2024-09-16T14:08:49.444239+0000","last_undegraded":"2024-09-16T14:08:49.444239+0000","last_fullsized":"2024-09-16T14:08:49.444239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:11.596666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"103'382","reported_seq":11604,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.457237+0000","last_change":"2024-09-16T14:07:22.350954+0000","last_active":"2024-09-16T14:08:51.457237+0000","last_peered":"2024-09-16T14:08:51.457237+0000","last_clean":"2024-09-16T14:08:51.457237+0000","last_became_active":"2024-09-16T14:07:22.350024+0000","last_became_peered":"2024-09-16T14:07:22.350024+0000","last_unstale":"2024-09-16T14:08:51.457237+0000","last_undegraded":"2024-09-16T14:08:51.457237+0000","last_fullsized":"2024-09-16T14:08:51.457237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:07.634077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002704522,"stat_sum":{"num_bytes":50255264,"num_objects":97,"num_object_clones":27,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":21,"num_read":11309,"num_read_kb":40812,"num_write":342,"num_write_kb":8733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":52,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444346+0000","last_change":"2024-09-16T14:08:28.215537+0000","last_active":"2024-09-16T14:08:49.444346+0000","last_peered":"2024-09-16T14:08:49.444346+0000","last_clean":"2024-09-16T14:08:49.444346+0000","last_became_active":"2024-09-16T14:08:28.214888+0000","last_became_peered":"2024-09-16T14:08:28.214888+0000","last_unstale":"2024-09-16T14:08:49.444346+0000","last_undegraded":"2024-09-16T14:08:49.444346+0000","last_fullsized":"2024-09-16T14:08:49.444346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:23.766012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":185,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.439855+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:08:50.439855+0000","last_peered":"2024-09-16T14:08:50.439855+0000","last_clean":"2024-09-16T14:08:50.439855+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:08:50.439855+0000","last_undegraded":"2024-09-16T14:08:50.439855+0000","last_fullsized":"2024-09-16T14:08:50.439855+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"103'454","reported_seq":2815,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.304133+0000","last_change":"2024-09-16T14:08:42.373173+0000","last_active":"2024-09-16T14:08:51.304133+0000","last_peered":"2024-09-16T14:08:51.304133+0000","last_clean":"2024-09-16T14:08:51.304133+0000","last_became_active":"2024-09-16T14:08:42.369933+0000","last_became_peered":"2024-09-16T14:08:42.369933+0000","last_unstale":"2024-09-16T14:08:51.304133+0000","last_undegraded":"2024-09-16T14:08:51.304133+0000","last_fullsized":"2024-09-16T14:08:51.304133+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:06.968050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022177080000000001,"stat_sum":{"num_bytes":20454053,"num_objects":104,"num_object_clones":19,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":30,"num_read":2742,"num_read_kb":10490,"num_write":527,"num_write_kb":5582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2056974,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"103'416","reported_seq":5477,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.103888+0000","last_change":"2024-09-16T14:08:30.360750+0000","last_active":"2024-09-16T14:08:51.103888+0000","last_peered":"2024-09-16T14:08:51.103888+0000","last_clean":"2024-09-16T14:08:51.103888+0000","last_became_active":"2024-09-16T14:08:30.357783+0000","last_became_peered":"2024-09-16T14:08:30.357783+0000","last_unstale":"2024-09-16T14:08:51.103888+0000","last_undegraded":"2024-09-16T14:08:51.103888+0000","last_fullsized":"2024-09-16T14:08:51.103888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:45.786220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339700000000001,"stat_sum":{"num_bytes":21348141,"num_objects":93,"num_object_clones":18,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":24,"num_read":6233,"num_read_kb":14774,"num_write":428,"num_write_kb":8650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2583563,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"103'574","reported_seq":5576,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.201348+0000","last_change":"2024-09-16T14:08:30.359428+0000","last_active":"2024-09-16T14:08:51.201348+0000","last_peered":"2024-09-16T14:08:51.201348+0000","last_clean":"2024-09-16T14:08:51.201348+0000","last_became_active":"2024-09-16T14:08:30.356626+0000","last_became_peered":"2024-09-16T14:08:30.356626+0000","last_unstale":"2024-09-16T14:08:51.201348+0000","last_undegraded":"2024-09-16T14:08:51.201348+0000","last_fullsized":"2024-09-16T14:08:51.201348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:50.638542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065751299999999998,"stat_sum":{"num_bytes":72490174,"num_objects":112,"num_object_clones":28,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":31,"num_read":6541,"num_read_kb":34388,"num_write":577,"num_write_kb":17863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":14526461,"num_keys_recovered":40,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"103'490","reported_seq":3636,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.304070+0000","last_change":"2024-09-16T14:08:41.367794+0000","last_active":"2024-09-16T14:08:51.304070+0000","last_peered":"2024-09-16T14:08:51.304070+0000","last_clean":"2024-09-16T14:08:51.304070+0000","last_became_active":"2024-09-16T14:08:41.366106+0000","last_became_peered":"2024-09-16T14:08:41.366106+0000","last_unstale":"2024-09-16T14:08:51.304070+0000","last_undegraded":"2024-09-16T14:08:51.304070+0000","last_fullsized":"2024-09-16T14:08:51.304070+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:15.221267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056925200000000002,"stat_sum":{"num_bytes":28458263,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":22,"num_read":5180,"num_read_kb":41136,"num_write":617,"num_write_kb":22311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":9737480,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"103'405","reported_seq":6468,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.201401+0000","last_change":"2024-09-16T14:08:41.365299+0000","last_active":"2024-09-16T14:08:51.201401+0000","last_peered":"2024-09-16T14:08:51.201401+0000","last_clean":"2024-09-16T14:08:51.201401+0000","last_became_active":"2024-09-16T14:08:41.363613+0000","last_became_peered":"2024-09-16T14:08:41.363613+0000","last_unstale":"2024-09-16T14:08:51.201401+0000","last_undegraded":"2024-09-16T14:08:51.201401+0000","last_fullsized":"2024-09-16T14:08:51.201401+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:22.795636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017881959999999999,"stat_sum":{"num_bytes":34724055,"num_objects":107,"num_object_clones":24,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":33,"num_read":6677,"num_read_kb":27124,"num_write":362,"num_write_kb":12159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":8807662,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"103'276","reported_seq":1408,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.790813+0000","last_change":"2024-09-16T14:08:28.444504+0000","last_active":"2024-09-16T14:08:50.790813+0000","last_peered":"2024-09-16T14:08:50.790813+0000","last_clean":"2024-09-16T14:08:50.790813+0000","last_became_active":"2024-09-16T14:08:28.220167+0000","last_became_peered":"2024-09-16T14:08:28.220167+0000","last_unstale":"2024-09-16T14:08:50.790813+0000","last_undegraded":"2024-09-16T14:08:50.790813+0000","last_fullsized":"2024-09-16T14:08:50.790813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:40.805140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074836900000000003,"stat_sum":{"num_bytes":32222075,"num_objects":85,"num_object_clones":19,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":19,"num_read":7263,"num_read_kb":18090,"num_write":238,"num_write_kb":5200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":5458762,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"103'248","reported_seq":3702,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.456514+0000","last_change":"2024-09-16T14:07:11.362189+0000","last_active":"2024-09-16T14:08:51.456514+0000","last_peered":"2024-09-16T14:08:51.456514+0000","last_clean":"2024-09-16T14:08:51.456514+0000","last_became_active":"2024-09-16T14:07:09.548973+0000","last_became_peered":"2024-09-16T14:07:09.548973+0000","last_unstale":"2024-09-16T14:08:51.456514+0000","last_undegraded":"2024-09-16T14:08:51.456514+0000","last_fullsized":"2024-09-16T14:08:51.456514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:17.345008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013505660000000001,"stat_sum":{"num_bytes":33212438,"num_objects":77,"num_object_clones":18,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":19,"num_read":4178,"num_read_kb":14461,"num_write":220,"num_write_kb":5422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"103'726","reported_seq":5334,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.185233+0000","last_change":"2024-09-16T14:07:11.342735+0000","last_active":"2024-09-16T14:08:51.185233+0000","last_peered":"2024-09-16T14:08:51.185233+0000","last_clean":"2024-09-16T14:08:51.185233+0000","last_became_active":"2024-09-16T14:07:09.559054+0000","last_became_peered":"2024-09-16T14:07:09.559054+0000","last_unstale":"2024-09-16T14:08:51.185233+0000","last_undegraded":"2024-09-16T14:08:51.185233+0000","last_fullsized":"2024-09-16T14:08:51.185233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.926445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372222,"stat_sum":{"num_bytes":43072047,"num_objects":91,"num_object_clones":15,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":18,"num_read":8705,"num_read_kb":18154,"num_write":988,"num_write_kb":9446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"103'444","reported_seq":11401,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.305256+0000","last_change":"2024-09-16T14:07:11.340733+0000","last_active":"2024-09-16T14:08:51.305256+0000","last_peered":"2024-09-16T14:08:51.305256+0000","last_clean":"2024-09-16T14:08:51.305256+0000","last_became_active":"2024-09-16T14:07:09.540341+0000","last_became_peered":"2024-09-16T14:07:09.540341+0000","last_unstale":"2024-09-16T14:08:51.305256+0000","last_undegraded":"2024-09-16T14:08:51.305256+0000","last_fullsized":"2024-09-16T14:08:51.305256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:18.953079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012161470000000001,"stat_sum":{"num_bytes":47759727,"num_objects":92,"num_object_clones":22,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":22,"num_read":12583,"num_read_kb":22519,"num_write":631,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"103'535","reported_seq":4236,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.080144+0000","last_change":"2024-09-16T14:08:32.368616+0000","last_active":"2024-09-16T14:08:51.080144+0000","last_peered":"2024-09-16T14:08:51.080144+0000","last_clean":"2024-09-16T14:08:51.080144+0000","last_became_active":"2024-09-16T14:08:32.366601+0000","last_became_peered":"2024-09-16T14:08:32.366601+0000","last_unstale":"2024-09-16T14:08:51.080144+0000","last_undegraded":"2024-09-16T14:08:51.080144+0000","last_fullsized":"2024-09-16T14:08:51.080144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:01.897916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037127800000000001,"stat_sum":{"num_bytes":21687881,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":18,"num_read":5048,"num_read_kb":12384,"num_write":703,"num_write_kb":8371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":635545,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"104'476","reported_seq":5117,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.447157+0000","last_change":"2024-09-16T14:08:30.363925+0000","last_active":"2024-09-16T14:08:51.447157+0000","last_peered":"2024-09-16T14:08:51.447157+0000","last_clean":"2024-09-16T14:08:51.447157+0000","last_became_active":"2024-09-16T14:08:30.360870+0000","last_became_peered":"2024-09-16T14:08:30.360870+0000","last_unstale":"2024-09-16T14:08:51.447157+0000","last_undegraded":"2024-09-16T14:08:51.447157+0000","last_fullsized":"2024-09-16T14:08:51.447157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:04.684260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012662420000000001,"stat_sum":{"num_bytes":60460437,"num_objects":93,"num_object_clones":17,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":23,"num_read":6830,"num_read_kb":54381,"num_write":513,"num_write_kb":28164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":11455145,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"103'381","reported_seq":6402,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.055698+0000","last_change":"2024-09-16T14:08:30.359625+0000","last_active":"2024-09-16T14:08:51.055698+0000","last_peered":"2024-09-16T14:08:51.055698+0000","last_clean":"2024-09-16T14:08:51.055698+0000","last_became_active":"2024-09-16T14:08:30.357685+0000","last_became_peered":"2024-09-16T14:08:30.357685+0000","last_unstale":"2024-09-16T14:08:51.055698+0000","last_undegraded":"2024-09-16T14:08:51.055698+0000","last_fullsized":"2024-09-16T14:08:51.055698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:26.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066482800000000001,"stat_sum":{"num_bytes":61287048,"num_objects":90,"num_object_clones":23,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":22,"num_read":7101,"num_read_kb":30682,"num_write":460,"num_write_kb":17613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":11054624,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1211184367,"num_objects":2935,"num_object_clones":636,"num_object_copies":5870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":735,"num_read":222469,"num_read_kb":752213,"num_write":23273,"num_write_kb":360808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1309,"num_bytes_recovered":89016121,"num_keys_recovered":147,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58269696,"data_stored":541873158,"data_compressed":14152731,"data_compressed_allocated":42991616,"data_compressed_original":533010998,"omap_allocated":8948,"internal_metadata":0},"log_size":18006,"ondisk_log_size":18006,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509468,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306176,"kb_used_data":8824,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065664,"statfs":{"total":96636764160,"available":96323239936,"internally_reserved":0,"allocated":9035776,"data_stored":90807840,"data_compressed":1800229,"data_compressed_allocated":6127616,"data_compressed_original":89320822,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.149},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.84099999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.95699999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.117},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.054}]}]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312508,"kb_used_data":6824,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94059332,"statfs":{"total":96636764160,"available":96316755968,"internally_reserved":0,"allocated":6987776,"data_stored":53914481,"data_compressed":1186335,"data_compressed_allocated":3956736,"data_compressed_original":52277115,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.97399999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.056}]},{"osd":7,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.878},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.91500000000000004}]}]},{"osd":3,"up_from":83,"seq":356482285574,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315548,"kb_used_data":12304,"kb_used_omap":31,"kb_used_meta":303200,"kb_avail":94056292,"statfs":{"total":96636764160,"available":96313643008,"internally_reserved":0,"allocated":12599296,"data_stored":98289687,"data_compressed":2749656,"data_compressed_allocated":8658944,"data_compressed_original":96122692,"omap_allocated":31864,"internal_metadata":310477704},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310852,"kb_used_data":9336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060988,"statfs":{"total":96636764160,"available":96318451712,"internally_reserved":0,"allocated":9560064,"data_stored":74057088,"data_compressed":2202008,"data_compressed_allocated":5746688,"data_compressed_original":71972677,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.435}]},{"osd":1,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82299999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.85999999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.80000000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310900,"kb_used_data":9380,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060940,"statfs":{"total":96636764160,"available":96318402560,"internally_reserved":0,"allocated":9605120,"data_stored":71529449,"data_compressed":1774741,"data_compressed_allocated":6230016,"data_compressed_original":69757720,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":0.44800000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.875},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.61599999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.94299999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.85699999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311496,"kb_used_data":9980,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060344,"statfs":{"total":96636764160,"available":96317792256,"internally_reserved":0,"allocated":10219520,"data_stored":85289588,"data_compressed":2489919,"data_compressed_allocated":6746112,"data_compressed_original":83386381,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.86199999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.79800000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.081},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.97799999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.01},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.401}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305404,"kb_used_data":8048,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066436,"statfs":{"total":96636764160,"available":96324030464,"internally_reserved":0,"allocated":8241152,"data_stored":70865241,"data_compressed":1849516,"data_compressed_allocated":5320704,"data_compressed_original":69317527,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.96099999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.091}]},{"osd":3,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.90500000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.499}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.1559999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.81699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9035776,"data_stored":84753892,"data_compressed":2490094,"data_compressed_allocated":6750208,"data_compressed_original":83394573,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8384512,"data_stored":70434311,"data_compressed":1795097,"data_compressed_allocated":6225920,"data_compressed_original":69233432,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8331264,"data_stored":73488095,"data_compressed":2202008,"data_compressed_allocated":5746688,"data_compressed_original":71972677,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11378688,"data_stored":97726417,"data_compressed":2749656,"data_compressed_allocated":8658944,"data_compressed_original":96122692,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7036928,"data_stored":70308246,"data_compressed":1849516,"data_compressed_allocated":5320704,"data_compressed_original":69317527,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5763072,"data_stored":52830524,"data_compressed":1188881,"data_compressed_allocated":3936256,"data_compressed_original":51752827,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7847936,"data_stored":90274699,"data_compressed":1813038,"data_compressed_allocated":6144000,"data_compressed_original":89341302,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":2056974,"data_compressed":64441,"data_compressed_allocated":208896,"data_compressed_original":1875968,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:08:54.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:08:54.882 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:54.880+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:54.922 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:54.923 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:54.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:54.977 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:54.970+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:55.031 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.124853287999997, 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.124830580000001, 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:08:55.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.042 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.124806427999999, 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:08:55.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:08:55.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.124773486999999, 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:08:55.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.052 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.124748803999999, 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:55.057 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.124725574999999, 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:08:55.062 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.122879611000002, 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:08:55.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:55.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.122836747000001, 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:08:55.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121377129000003, 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:08:55.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.078 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121361360000002, 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:55.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:55.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:08:55.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121345607999999, 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:08:55.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:08:55.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.088 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121330176999997, 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:08:55.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:08:55.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:08:55.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:08:55.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121314538999997, 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:08:55.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:55.099 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121299399999998, 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:08:55.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:55.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121284563000003, 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:08:55.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:08:55.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.110 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121261558999997, 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:08:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:55.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:08:55.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:08:55.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121246333999999, 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:08:55.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:08:55.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.12123115, 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:08:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:08:55.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121215905, 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.128 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:08:55.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:08:55.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.121200747000003, 2024-09-16T14:08:55.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:08:55.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.138 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:55.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:55.140 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:55.136+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:55.166 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:08:55.168 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:08:55.178 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-16T14:08:54.797524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1211774754,"num_objects":2939,"num_object_clones":636,"num_object_copies":5878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2939,"num_whiteouts":735,"num_read":222535,"num_read_kb":752268,"num_write":23376,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1309,"num_bytes_recovered":89016121,"num_keys_recovered":147,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18081,"ondisk_log_size":18081,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2172884,"kb_used_data":64696,"kb_used_omap":41,"kb_used_meta":2108054,"kb_avail":658429996,"statfs":{"total":676457349120,"available":674232315904,"internally_reserved":0,"allocated":66248704,"data_stored":544753374,"data_compressed":14052404,"data_compressed_allocated":42786816,"data_compressed_original":532154934,"omap_allocated":42088,"internal_metadata":2158648216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54465830,"num_objects":117,"num_object_clones":30,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":46,"num_read":9939,"num_read_kb":30476,"num_write":658,"num_write_kb":4693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000697"},"pg_stats":[{"pgid":"3.1f","version":"103'1637","reported_seq":6463,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.412759+0000","last_change":"2024-09-16T14:07:11.338271+0000","last_active":"2024-09-16T14:08:51.412759+0000","last_peered":"2024-09-16T14:08:51.412759+0000","last_clean":"2024-09-16T14:08:51.412759+0000","last_became_active":"2024-09-16T14:07:09.540900+0000","last_became_peered":"2024-09-16T14:07:09.540900+0000","last_unstale":"2024-09-16T14:08:51.412759+0000","last_undegraded":"2024-09-16T14:08:51.412759+0000","last_fullsized":"2024-09-16T14:08:51.412759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:59.820596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233770000000001,"stat_sum":{"num_bytes":22853786,"num_objects":83,"num_object_clones":15,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":15,"num_read":14341,"num_read_kb":14697,"num_write":2790,"num_write_kb":6574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"104'589","reported_seq":2048,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.446724+0000","last_change":"2024-09-16T14:07:11.337654+0000","last_active":"2024-09-16T14:08:51.446724+0000","last_peered":"2024-09-16T14:08:51.446724+0000","last_clean":"2024-09-16T14:08:51.446724+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:08:51.446724+0000","last_undegraded":"2024-09-16T14:08:51.446724+0000","last_fullsized":"2024-09-16T14:08:51.446724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060911299999999999,"stat_sum":{"num_bytes":30496472,"num_objects":92,"num_object_clones":19,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":18,"num_read":3161,"num_read_kb":19744,"num_write":820,"num_write_kb":9189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"103'380","reported_seq":2827,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.970141+0000","last_change":"2024-09-16T14:08:28.327780+0000","last_active":"2024-09-16T14:08:50.970141+0000","last_peered":"2024-09-16T14:08:50.970141+0000","last_clean":"2024-09-16T14:08:50.970141+0000","last_became_active":"2024-09-16T14:08:28.215309+0000","last_became_peered":"2024-09-16T14:08:28.215309+0000","last_unstale":"2024-09-16T14:08:50.970141+0000","last_undegraded":"2024-09-16T14:08:50.970141+0000","last_fullsized":"2024-09-16T14:08:50.970141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:42.429361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072576699999999999,"stat_sum":{"num_bytes":35735733,"num_objects":100,"num_object_clones":21,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":21,"num_read":6078,"num_read_kb":21111,"num_write":553,"num_write_kb":12718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":5718086,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"103'719","reported_seq":13144,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.454612+0000","last_change":"2024-09-16T14:07:11.340416+0000","last_active":"2024-09-16T14:08:51.454612+0000","last_peered":"2024-09-16T14:08:51.454612+0000","last_clean":"2024-09-16T14:08:51.454612+0000","last_became_active":"2024-09-16T14:07:09.551657+0000","last_became_peered":"2024-09-16T14:07:09.551657+0000","last_unstale":"2024-09-16T14:08:51.454612+0000","last_undegraded":"2024-09-16T14:08:51.454612+0000","last_fullsized":"2024-09-16T14:08:51.454612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:49.635426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064309800000000004,"stat_sum":{"num_bytes":38411660,"num_objects":88,"num_object_clones":23,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":27,"num_read":14608,"num_read_kb":24332,"num_write":1009,"num_write_kb":8382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"103'430","reported_seq":4433,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.455995+0000","last_change":"2024-09-16T14:07:22.350217+0000","last_active":"2024-09-16T14:08:51.455995+0000","last_peered":"2024-09-16T14:08:51.455995+0000","last_clean":"2024-09-16T14:08:51.455995+0000","last_became_active":"2024-09-16T14:07:22.349546+0000","last_became_peered":"2024-09-16T14:07:22.349546+0000","last_unstale":"2024-09-16T14:08:51.455995+0000","last_undegraded":"2024-09-16T14:08:51.455995+0000","last_fullsized":"2024-09-16T14:08:51.455995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:30.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021275310000000002,"stat_sum":{"num_bytes":23725811,"num_objects":91,"num_object_clones":16,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":23,"num_read":6258,"num_read_kb":15357,"num_write":486,"num_write_kb":6241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"103'432","reported_seq":4953,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.454504+0000","last_change":"2024-09-16T14:07:11.340816+0000","last_active":"2024-09-16T14:08:51.454504+0000","last_peered":"2024-09-16T14:08:51.454504+0000","last_clean":"2024-09-16T14:08:51.454504+0000","last_became_active":"2024-09-16T14:07:09.540237+0000","last_became_peered":"2024-09-16T14:07:09.540237+0000","last_unstale":"2024-09-16T14:08:51.454504+0000","last_undegraded":"2024-09-16T14:08:51.454504+0000","last_fullsized":"2024-09-16T14:08:51.454504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:38.147579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099540499999999999,"stat_sum":{"num_bytes":22383318,"num_objects":85,"num_object_clones":20,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":23,"num_read":5359,"num_read_kb":23938,"num_write":444,"num_write_kb":21969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"104'407","reported_seq":1118,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.441574+0000","last_change":"2024-09-16T14:07:22.346713+0000","last_active":"2024-09-16T14:08:51.441574+0000","last_peered":"2024-09-16T14:08:51.441574+0000","last_clean":"2024-09-16T14:08:51.441574+0000","last_became_active":"2024-09-16T14:07:22.346031+0000","last_became_peered":"2024-09-16T14:07:22.346031+0000","last_unstale":"2024-09-16T14:08:51.441574+0000","last_undegraded":"2024-09-16T14:08:51.441574+0000","last_fullsized":"2024-09-16T14:08:51.441574+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:39.265638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003350142,"stat_sum":{"num_bytes":31595139,"num_objects":84,"num_object_clones":14,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":16,"num_read":1341,"num_read_kb":21935,"num_write":502,"num_write_kb":17534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"104'474","reported_seq":5207,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.469005+0000","last_change":"2024-09-16T14:07:11.338172+0000","last_active":"2024-09-16T14:08:51.469005+0000","last_peered":"2024-09-16T14:08:51.469005+0000","last_clean":"2024-09-16T14:08:51.469005+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:08:51.469005+0000","last_undegraded":"2024-09-16T14:08:51.469005+0000","last_fullsized":"2024-09-16T14:08:51.469005+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148865,"stat_sum":{"num_bytes":37104369,"num_objects":71,"num_object_clones":12,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":15,"num_read":6916,"num_read_kb":19923,"num_write":525,"num_write_kb":7568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"103'1147","reported_seq":4513,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.182998+0000","last_change":"2024-09-16T14:08:31.367987+0000","last_active":"2024-09-16T14:08:51.182998+0000","last_peered":"2024-09-16T14:08:51.182998+0000","last_clean":"2024-09-16T14:08:51.182998+0000","last_became_active":"2024-09-16T14:08:31.364152+0000","last_became_peered":"2024-09-16T14:08:31.364152+0000","last_unstale":"2024-09-16T14:08:51.182998+0000","last_undegraded":"2024-09-16T14:08:51.182998+0000","last_fullsized":"2024-09-16T14:08:51.182998+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:05.530637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001215788,"stat_sum":{"num_bytes":31074832,"num_objects":89,"num_object_clones":21,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":26,"num_read":8110,"num_read_kb":32761,"num_write":1847,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1214676,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"102'453","reported_seq":6626,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.459065+0000","last_change":"2024-09-16T14:08:32.368069+0000","last_active":"2024-09-16T14:08:51.459065+0000","last_peered":"2024-09-16T14:08:51.459065+0000","last_clean":"2024-09-16T14:08:51.459065+0000","last_became_active":"2024-09-16T14:08:32.365885+0000","last_became_peered":"2024-09-16T14:08:32.365885+0000","last_unstale":"2024-09-16T14:08:51.459065+0000","last_undegraded":"2024-09-16T14:08:51.459065+0000","last_fullsized":"2024-09-16T14:08:51.459065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:53.426423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500454,"stat_sum":{"num_bytes":45180005,"num_objects":105,"num_object_clones":23,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":27,"num_read":7555,"num_read_kb":25286,"num_write":477,"num_write_kb":9154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":3589265,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"104'310","reported_seq":3954,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.450858+0000","last_change":"2024-09-16T14:07:11.343447+0000","last_active":"2024-09-16T14:08:51.450858+0000","last_peered":"2024-09-16T14:08:51.450858+0000","last_clean":"2024-09-16T14:08:51.450858+0000","last_became_active":"2024-09-16T14:07:09.549545+0000","last_became_peered":"2024-09-16T14:07:09.549545+0000","last_unstale":"2024-09-16T14:08:51.450858+0000","last_undegraded":"2024-09-16T14:08:51.450858+0000","last_fullsized":"2024-09-16T14:08:51.450858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:27.229897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012153120000000001,"stat_sum":{"num_bytes":35475058,"num_objects":96,"num_object_clones":29,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":30,"num_read":3686,"num_read_kb":23719,"num_write":332,"num_write_kb":6356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"103'472","reported_seq":3871,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.196472+0000","last_change":"2024-09-16T14:08:30.361920+0000","last_active":"2024-09-16T14:08:51.196472+0000","last_peered":"2024-09-16T14:08:51.196472+0000","last_clean":"2024-09-16T14:08:51.196472+0000","last_became_active":"2024-09-16T14:08:30.360514+0000","last_became_peered":"2024-09-16T14:08:30.360514+0000","last_unstale":"2024-09-16T14:08:51.196472+0000","last_undegraded":"2024-09-16T14:08:51.196472+0000","last_fullsized":"2024-09-16T14:08:51.196472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:15.505375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009984429999999999,"stat_sum":{"num_bytes":44661799,"num_objects":88,"num_object_clones":18,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":19,"num_read":5437,"num_read_kb":25061,"num_write":505,"num_write_kb":7261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":1548718,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444450+0000","last_change":"2024-09-16T14:08:28.215419+0000","last_active":"2024-09-16T14:08:49.444450+0000","last_peered":"2024-09-16T14:08:49.444450+0000","last_clean":"2024-09-16T14:08:49.444450+0000","last_became_active":"2024-09-16T14:08:28.215112+0000","last_became_peered":"2024-09-16T14:08:28.215112+0000","last_unstale":"2024-09-16T14:08:49.444450+0000","last_undegraded":"2024-09-16T14:08:49.444450+0000","last_fullsized":"2024-09-16T14:08:49.444450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:19.841089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"103'565","reported_seq":3827,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.575054+0000","last_change":"2024-09-16T14:08:41.366677+0000","last_active":"2024-09-16T14:08:50.575054+0000","last_peered":"2024-09-16T14:08:50.575054+0000","last_clean":"2024-09-16T14:08:50.575054+0000","last_became_active":"2024-09-16T14:08:41.361496+0000","last_became_peered":"2024-09-16T14:08:41.361496+0000","last_unstale":"2024-09-16T14:08:50.575054+0000","last_undegraded":"2024-09-16T14:08:50.575054+0000","last_fullsized":"2024-09-16T14:08:50.575054+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:12.420908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032490359999999999,"stat_sum":{"num_bytes":43515179,"num_objects":95,"num_object_clones":19,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":16,"num_read":4633,"num_read_kb":21891,"num_write":704,"num_write_kb":8605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":3405893,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.496070+0000","last_change":"2024-09-16T14:07:22.350069+0000","last_active":"2024-09-16T14:08:50.496070+0000","last_peered":"2024-09-16T14:08:50.496070+0000","last_clean":"2024-09-16T14:08:50.496070+0000","last_became_active":"2024-09-16T14:07:22.349757+0000","last_became_peered":"2024-09-16T14:07:22.349757+0000","last_unstale":"2024-09-16T14:08:50.496070+0000","last_undegraded":"2024-09-16T14:08:50.496070+0000","last_fullsized":"2024-09-16T14:08:50.496070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:57.826775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"103'741","reported_seq":6400,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.304500+0000","last_change":"2024-09-16T14:07:22.351951+0000","last_active":"2024-09-16T14:08:51.304500+0000","last_peered":"2024-09-16T14:08:51.304500+0000","last_clean":"2024-09-16T14:08:51.304500+0000","last_became_active":"2024-09-16T14:07:22.351394+0000","last_became_peered":"2024-09-16T14:07:22.351394+0000","last_unstale":"2024-09-16T14:08:51.304500+0000","last_undegraded":"2024-09-16T14:08:51.304500+0000","last_fullsized":"2024-09-16T14:08:51.304500+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:55.945287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027484229999999998,"stat_sum":{"num_bytes":76394086,"num_objects":93,"num_object_clones":20,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":26,"num_read":9079,"num_read_kb":41824,"num_write":1091,"num_write_kb":28785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444443+0000","last_change":"2024-09-16T14:08:28.213074+0000","last_active":"2024-09-16T14:08:49.444443+0000","last_peered":"2024-09-16T14:08:49.444443+0000","last_clean":"2024-09-16T14:08:49.444443+0000","last_became_active":"2024-09-16T14:08:28.212640+0000","last_became_peered":"2024-09-16T14:08:28.212640+0000","last_unstale":"2024-09-16T14:08:49.444443+0000","last_undegraded":"2024-09-16T14:08:49.444443+0000","last_fullsized":"2024-09-16T14:08:49.444443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:09.532319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"103'393","reported_seq":3103,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.344959+0000","last_change":"2024-09-16T14:07:11.547417+0000","last_active":"2024-09-16T14:08:51.344959+0000","last_peered":"2024-09-16T14:08:51.344959+0000","last_clean":"2024-09-16T14:08:51.344959+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:08:51.344959+0000","last_undegraded":"2024-09-16T14:08:51.344959+0000","last_fullsized":"2024-09-16T14:08:51.344959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097608199999999997,"stat_sum":{"num_bytes":15659686,"num_objects":88,"num_object_clones":20,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":26,"num_read":3960,"num_read_kb":12390,"num_write":424,"num_write_kb":5434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.442372+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:08:50.442372+0000","last_peered":"2024-09-16T14:08:50.442372+0000","last_clean":"2024-09-16T14:08:50.442372+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:08:50.442372+0000","last_undegraded":"2024-09-16T14:08:50.442372+0000","last_fullsized":"2024-09-16T14:08:50.442372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"103'544","reported_seq":8558,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.207546+0000","last_change":"2024-09-16T14:07:11.361331+0000","last_active":"2024-09-16T14:08:51.207546+0000","last_peered":"2024-09-16T14:08:51.207546+0000","last_clean":"2024-09-16T14:08:51.207546+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:08:51.207546+0000","last_undegraded":"2024-09-16T14:08:51.207546+0000","last_fullsized":"2024-09-16T14:08:51.207546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062220600000000004,"stat_sum":{"num_bytes":26234909,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":23,"num_read":9302,"num_read_kb":11175,"num_write":684,"num_write_kb":4642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"103'896","reported_seq":3672,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.203934+0000","last_change":"2024-09-16T14:07:11.342816+0000","last_active":"2024-09-16T14:08:51.203934+0000","last_peered":"2024-09-16T14:08:51.203934+0000","last_clean":"2024-09-16T14:08:51.203934+0000","last_became_active":"2024-09-16T14:07:09.549842+0000","last_became_peered":"2024-09-16T14:07:09.549842+0000","last_unstale":"2024-09-16T14:08:51.203934+0000","last_undegraded":"2024-09-16T14:08:51.203934+0000","last_fullsized":"2024-09-16T14:08:51.203934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:47.724999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014467779999999999,"stat_sum":{"num_bytes":37384569,"num_objects":95,"num_object_clones":18,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":7877,"num_read_kb":16086,"num_write":1175,"num_write_kb":9291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":53,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444325+0000","last_change":"2024-09-16T14:08:28.217645+0000","last_active":"2024-09-16T14:08:49.444325+0000","last_peered":"2024-09-16T14:08:49.444325+0000","last_clean":"2024-09-16T14:08:49.444325+0000","last_became_active":"2024-09-16T14:08:28.217418+0000","last_became_peered":"2024-09-16T14:08:28.217418+0000","last_unstale":"2024-09-16T14:08:49.444325+0000","last_undegraded":"2024-09-16T14:08:49.444325+0000","last_fullsized":"2024-09-16T14:08:49.444325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:25.918964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"103'1008","reported_seq":4527,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.475021+0000","last_change":"2024-09-16T14:08:41.367878+0000","last_active":"2024-09-16T14:08:51.475021+0000","last_peered":"2024-09-16T14:08:51.475021+0000","last_clean":"2024-09-16T14:08:51.475021+0000","last_became_active":"2024-09-16T14:08:41.365767+0000","last_became_peered":"2024-09-16T14:08:41.365767+0000","last_unstale":"2024-09-16T14:08:51.475021+0000","last_undegraded":"2024-09-16T14:08:51.475021+0000","last_fullsized":"2024-09-16T14:08:51.475021+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:19:25.019511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076943300000000003,"stat_sum":{"num_bytes":51051893,"num_objects":112,"num_object_clones":26,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":31,"num_read":6811,"num_read_kb":21948,"num_write":1508,"num_write_kb":9583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":7223267,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.442093+0000","last_change":"2024-09-16T14:08:38.421227+0000","last_active":"2024-09-16T14:08:50.442093+0000","last_peered":"2024-09-16T14:08:50.442093+0000","last_clean":"2024-09-16T14:08:50.442093+0000","last_became_active":"2024-09-16T14:08:38.420867+0000","last_became_peered":"2024-09-16T14:08:38.420867+0000","last_unstale":"2024-09-16T14:08:50.442093+0000","last_undegraded":"2024-09-16T14:08:50.442093+0000","last_fullsized":"2024-09-16T14:08:50.442093+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:52.563336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"104'602","reported_seq":6178,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.447233+0000","last_change":"2024-09-16T14:07:11.547270+0000","last_active":"2024-09-16T14:08:51.447233+0000","last_peered":"2024-09-16T14:08:51.447233+0000","last_clean":"2024-09-16T14:08:51.447233+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:08:51.447233+0000","last_undegraded":"2024-09-16T14:08:51.447233+0000","last_fullsized":"2024-09-16T14:08:51.447233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085373199999999995,"stat_sum":{"num_bytes":34814460,"num_objects":89,"num_object_clones":17,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":24,"num_read":7567,"num_read_kb":19640,"num_write":791,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444239+0000","last_change":"2024-09-16T14:08:28.214757+0000","last_active":"2024-09-16T14:08:49.444239+0000","last_peered":"2024-09-16T14:08:49.444239+0000","last_clean":"2024-09-16T14:08:49.444239+0000","last_became_active":"2024-09-16T14:08:28.213488+0000","last_became_peered":"2024-09-16T14:08:28.213488+0000","last_unstale":"2024-09-16T14:08:49.444239+0000","last_undegraded":"2024-09-16T14:08:49.444239+0000","last_fullsized":"2024-09-16T14:08:49.444239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:11.596666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"103'382","reported_seq":11604,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.457237+0000","last_change":"2024-09-16T14:07:22.350954+0000","last_active":"2024-09-16T14:08:51.457237+0000","last_peered":"2024-09-16T14:08:51.457237+0000","last_clean":"2024-09-16T14:08:51.457237+0000","last_became_active":"2024-09-16T14:07:22.350024+0000","last_became_peered":"2024-09-16T14:07:22.350024+0000","last_unstale":"2024-09-16T14:08:51.457237+0000","last_undegraded":"2024-09-16T14:08:51.457237+0000","last_fullsized":"2024-09-16T14:08:51.457237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:07.634077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002704522,"stat_sum":{"num_bytes":50255264,"num_objects":97,"num_object_clones":27,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":21,"num_read":11309,"num_read_kb":40812,"num_write":342,"num_write_kb":8733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":52,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T14:08:49.444346+0000","last_change":"2024-09-16T14:08:28.215537+0000","last_active":"2024-09-16T14:08:49.444346+0000","last_peered":"2024-09-16T14:08:49.444346+0000","last_clean":"2024-09-16T14:08:49.444346+0000","last_became_active":"2024-09-16T14:08:28.214888+0000","last_became_peered":"2024-09-16T14:08:28.214888+0000","last_unstale":"2024-09-16T14:08:49.444346+0000","last_undegraded":"2024-09-16T14:08:49.444346+0000","last_fullsized":"2024-09-16T14:08:49.444346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:23.766012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":185,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.439855+0000","last_change":"2024-09-16T14:06:53.995334+0000","last_active":"2024-09-16T14:08:50.439855+0000","last_peered":"2024-09-16T14:08:50.439855+0000","last_clean":"2024-09-16T14:08:50.439855+0000","last_became_active":"2024-09-16T14:06:53.993128+0000","last_became_peered":"2024-09-16T14:06:53.993128+0000","last_unstale":"2024-09-16T14:08:50.439855+0000","last_undegraded":"2024-09-16T14:08:50.439855+0000","last_fullsized":"2024-09-16T14:08:50.439855+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:16.649295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"103'454","reported_seq":2815,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.304133+0000","last_change":"2024-09-16T14:08:42.373173+0000","last_active":"2024-09-16T14:08:51.304133+0000","last_peered":"2024-09-16T14:08:51.304133+0000","last_clean":"2024-09-16T14:08:51.304133+0000","last_became_active":"2024-09-16T14:08:42.369933+0000","last_became_peered":"2024-09-16T14:08:42.369933+0000","last_unstale":"2024-09-16T14:08:51.304133+0000","last_undegraded":"2024-09-16T14:08:51.304133+0000","last_fullsized":"2024-09-16T14:08:51.304133+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:06.968050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022177080000000001,"stat_sum":{"num_bytes":20454053,"num_objects":104,"num_object_clones":19,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":30,"num_read":2742,"num_read_kb":10490,"num_write":527,"num_write_kb":5582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2056974,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"103'416","reported_seq":5477,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.103888+0000","last_change":"2024-09-16T14:08:30.360750+0000","last_active":"2024-09-16T14:08:51.103888+0000","last_peered":"2024-09-16T14:08:51.103888+0000","last_clean":"2024-09-16T14:08:51.103888+0000","last_became_active":"2024-09-16T14:08:30.357783+0000","last_became_peered":"2024-09-16T14:08:30.357783+0000","last_unstale":"2024-09-16T14:08:51.103888+0000","last_undegraded":"2024-09-16T14:08:51.103888+0000","last_fullsized":"2024-09-16T14:08:51.103888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:45.786220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057339700000000001,"stat_sum":{"num_bytes":21348141,"num_objects":93,"num_object_clones":18,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":24,"num_read":6233,"num_read_kb":14774,"num_write":428,"num_write_kb":8650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":2583563,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"103'574","reported_seq":5576,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.201348+0000","last_change":"2024-09-16T14:08:30.359428+0000","last_active":"2024-09-16T14:08:51.201348+0000","last_peered":"2024-09-16T14:08:51.201348+0000","last_clean":"2024-09-16T14:08:51.201348+0000","last_became_active":"2024-09-16T14:08:30.356626+0000","last_became_peered":"2024-09-16T14:08:30.356626+0000","last_unstale":"2024-09-16T14:08:51.201348+0000","last_undegraded":"2024-09-16T14:08:51.201348+0000","last_fullsized":"2024-09-16T14:08:51.201348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:50.638542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065751299999999998,"stat_sum":{"num_bytes":72490174,"num_objects":112,"num_object_clones":28,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":31,"num_read":6541,"num_read_kb":34388,"num_write":577,"num_write_kb":17863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":14526461,"num_keys_recovered":40,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"103'490","reported_seq":3636,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.304070+0000","last_change":"2024-09-16T14:08:41.367794+0000","last_active":"2024-09-16T14:08:51.304070+0000","last_peered":"2024-09-16T14:08:51.304070+0000","last_clean":"2024-09-16T14:08:51.304070+0000","last_became_active":"2024-09-16T14:08:41.366106+0000","last_became_peered":"2024-09-16T14:08:41.366106+0000","last_unstale":"2024-09-16T14:08:51.304070+0000","last_undegraded":"2024-09-16T14:08:51.304070+0000","last_fullsized":"2024-09-16T14:08:51.304070+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:15.221267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056925200000000002,"stat_sum":{"num_bytes":28458263,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":22,"num_read":5180,"num_read_kb":41136,"num_write":617,"num_write_kb":22311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":9737480,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"103'405","reported_seq":6468,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.201401+0000","last_change":"2024-09-16T14:08:41.365299+0000","last_active":"2024-09-16T14:08:51.201401+0000","last_peered":"2024-09-16T14:08:51.201401+0000","last_clean":"2024-09-16T14:08:51.201401+0000","last_became_active":"2024-09-16T14:08:41.363613+0000","last_became_peered":"2024-09-16T14:08:41.363613+0000","last_unstale":"2024-09-16T14:08:51.201401+0000","last_undegraded":"2024-09-16T14:08:51.201401+0000","last_fullsized":"2024-09-16T14:08:51.201401+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:22.795636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017881959999999999,"stat_sum":{"num_bytes":34724055,"num_objects":107,"num_object_clones":24,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":33,"num_read":6677,"num_read_kb":27124,"num_write":362,"num_write_kb":12159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":8807662,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"103'276","reported_seq":1408,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:50.790813+0000","last_change":"2024-09-16T14:08:28.444504+0000","last_active":"2024-09-16T14:08:50.790813+0000","last_peered":"2024-09-16T14:08:50.790813+0000","last_clean":"2024-09-16T14:08:50.790813+0000","last_became_active":"2024-09-16T14:08:28.220167+0000","last_became_peered":"2024-09-16T14:08:28.220167+0000","last_unstale":"2024-09-16T14:08:50.790813+0000","last_undegraded":"2024-09-16T14:08:50.790813+0000","last_fullsized":"2024-09-16T14:08:50.790813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:40.805140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074836900000000003,"stat_sum":{"num_bytes":32222075,"num_objects":85,"num_object_clones":19,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":19,"num_read":7263,"num_read_kb":18090,"num_write":238,"num_write_kb":5200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":5458762,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"103'248","reported_seq":3702,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.456514+0000","last_change":"2024-09-16T14:07:11.362189+0000","last_active":"2024-09-16T14:08:51.456514+0000","last_peered":"2024-09-16T14:08:51.456514+0000","last_clean":"2024-09-16T14:08:51.456514+0000","last_became_active":"2024-09-16T14:07:09.548973+0000","last_became_peered":"2024-09-16T14:07:09.548973+0000","last_unstale":"2024-09-16T14:08:51.456514+0000","last_undegraded":"2024-09-16T14:08:51.456514+0000","last_fullsized":"2024-09-16T14:08:51.456514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:17.345008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013505660000000001,"stat_sum":{"num_bytes":33212438,"num_objects":77,"num_object_clones":18,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":19,"num_read":4178,"num_read_kb":14461,"num_write":220,"num_write_kb":5422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"103'726","reported_seq":5334,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.185233+0000","last_change":"2024-09-16T14:07:11.342735+0000","last_active":"2024-09-16T14:08:51.185233+0000","last_peered":"2024-09-16T14:08:51.185233+0000","last_clean":"2024-09-16T14:08:51.185233+0000","last_became_active":"2024-09-16T14:07:09.559054+0000","last_became_peered":"2024-09-16T14:07:09.559054+0000","last_unstale":"2024-09-16T14:08:51.185233+0000","last_undegraded":"2024-09-16T14:08:51.185233+0000","last_fullsized":"2024-09-16T14:08:51.185233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:59.926445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372222,"stat_sum":{"num_bytes":43072047,"num_objects":91,"num_object_clones":15,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":18,"num_read":8705,"num_read_kb":18154,"num_write":988,"num_write_kb":9446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"103'444","reported_seq":11401,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.305256+0000","last_change":"2024-09-16T14:07:11.340733+0000","last_active":"2024-09-16T14:08:51.305256+0000","last_peered":"2024-09-16T14:08:51.305256+0000","last_clean":"2024-09-16T14:08:51.305256+0000","last_became_active":"2024-09-16T14:07:09.540341+0000","last_became_peered":"2024-09-16T14:07:09.540341+0000","last_unstale":"2024-09-16T14:08:51.305256+0000","last_undegraded":"2024-09-16T14:08:51.305256+0000","last_fullsized":"2024-09-16T14:08:51.305256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:18.953079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012161470000000001,"stat_sum":{"num_bytes":47759727,"num_objects":92,"num_object_clones":22,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":22,"num_read":12583,"num_read_kb":22519,"num_write":631,"num_write_kb":14222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"103'535","reported_seq":4236,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.080144+0000","last_change":"2024-09-16T14:08:32.368616+0000","last_active":"2024-09-16T14:08:51.080144+0000","last_peered":"2024-09-16T14:08:51.080144+0000","last_clean":"2024-09-16T14:08:51.080144+0000","last_became_active":"2024-09-16T14:08:32.366601+0000","last_became_peered":"2024-09-16T14:08:32.366601+0000","last_unstale":"2024-09-16T14:08:51.080144+0000","last_undegraded":"2024-09-16T14:08:51.080144+0000","last_fullsized":"2024-09-16T14:08:51.080144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:01.897916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037127800000000001,"stat_sum":{"num_bytes":21687881,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":18,"num_read":5048,"num_read_kb":12384,"num_write":703,"num_write_kb":8371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":635545,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"104'476","reported_seq":5117,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.447157+0000","last_change":"2024-09-16T14:08:30.363925+0000","last_active":"2024-09-16T14:08:51.447157+0000","last_peered":"2024-09-16T14:08:51.447157+0000","last_clean":"2024-09-16T14:08:51.447157+0000","last_became_active":"2024-09-16T14:08:30.360870+0000","last_became_peered":"2024-09-16T14:08:30.360870+0000","last_unstale":"2024-09-16T14:08:51.447157+0000","last_undegraded":"2024-09-16T14:08:51.447157+0000","last_fullsized":"2024-09-16T14:08:51.447157+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:04.684260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012662420000000001,"stat_sum":{"num_bytes":60460437,"num_objects":93,"num_object_clones":17,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":23,"num_read":6830,"num_read_kb":54381,"num_write":513,"num_write_kb":28164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":11455145,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"103'381","reported_seq":6402,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:08:51.055698+0000","last_change":"2024-09-16T14:08:30.359625+0000","last_active":"2024-09-16T14:08:51.055698+0000","last_peered":"2024-09-16T14:08:51.055698+0000","last_clean":"2024-09-16T14:08:51.055698+0000","last_became_active":"2024-09-16T14:08:30.357685+0000","last_became_peered":"2024-09-16T14:08:30.357685+0000","last_unstale":"2024-09-16T14:08:51.055698+0000","last_undegraded":"2024-09-16T14:08:51.055698+0000","last_fullsized":"2024-09-16T14:08:51.055698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:26.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066482800000000001,"stat_sum":{"num_bytes":61287048,"num_objects":90,"num_object_clones":23,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":22,"num_read":7101,"num_read_kb":30682,"num_write":460,"num_write_kb":17613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":11054624,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1211184367,"num_objects":2935,"num_object_clones":636,"num_object_copies":5870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":735,"num_read":222469,"num_read_kb":752213,"num_write":23273,"num_write_kb":360808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1309,"num_bytes_recovered":89016121,"num_keys_recovered":147,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58269696,"data_stored":541873158,"data_compressed":14152731,"data_compressed_allocated":42991616,"data_compressed_original":533010998,"omap_allocated":8948,"internal_metadata":0},"log_size":18006,"ondisk_log_size":18006,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10782,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509468,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306176,"kb_used_data":8824,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065664,"statfs":{"total":96636764160,"available":96323239936,"internally_reserved":0,"allocated":9035776,"data_stored":90807840,"data_compressed":1800229,"data_compressed_allocated":6127616,"data_compressed_original":89320822,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.149},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.84099999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.95699999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.117},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.054}]}]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312508,"kb_used_data":6824,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94059332,"statfs":{"total":96636764160,"available":96316755968,"internally_reserved":0,"allocated":6987776,"data_stored":53914481,"data_compressed":1186335,"data_compressed_allocated":3956736,"data_compressed_original":52277115,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.97399999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.056}]},{"osd":7,"last update":"Mon Sep 16 14:07:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.878},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.91500000000000004}]}]},{"osd":3,"up_from":83,"seq":356482285574,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315548,"kb_used_data":12304,"kb_used_omap":31,"kb_used_meta":303200,"kb_avail":94056292,"statfs":{"total":96636764160,"available":96313643008,"internally_reserved":0,"allocated":12599296,"data_stored":98289687,"data_compressed":2749656,"data_compressed_allocated":8658944,"data_compressed_original":96122692,"omap_allocated":31864,"internal_metadata":310477704},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310852,"kb_used_data":9336,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060988,"statfs":{"total":96636764160,"available":96318451712,"internally_reserved":0,"allocated":9560064,"data_stored":74057088,"data_compressed":2202008,"data_compressed_allocated":5746688,"data_compressed_original":71972677,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.435}]},{"osd":1,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82299999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.85999999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:07:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.80000000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310900,"kb_used_data":9380,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060940,"statfs":{"total":96636764160,"available":96318402560,"internally_reserved":0,"allocated":9605120,"data_stored":71529449,"data_compressed":1774741,"data_compressed_allocated":6230016,"data_compressed_original":69757720,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.6899999999999999,"5min":2.6899999999999999,"15min":2.6899999999999999},"last":0.44800000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.875},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.61599999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.94299999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.85699999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311496,"kb_used_data":9980,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94060344,"statfs":{"total":96636764160,"available":96317792256,"internally_reserved":0,"allocated":10219520,"data_stored":85289588,"data_compressed":2489919,"data_compressed_allocated":6746112,"data_compressed_original":83386381,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.86199999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.5230000000000001,"5min":2.5230000000000001,"15min":2.5230000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.79800000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.081},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.97799999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:07:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.01},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":1.401}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305404,"kb_used_data":8048,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066436,"statfs":{"total":96636764160,"available":96324030464,"internally_reserved":0,"allocated":8241152,"data_stored":70865241,"data_compressed":1849516,"data_compressed_allocated":5320704,"data_compressed_original":69317527,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.96099999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.091}]},{"osd":3,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.90500000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.499}]},{"osd":6,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.1559999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:07:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.81699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9035776,"data_stored":84753892,"data_compressed":2490094,"data_compressed_allocated":6750208,"data_compressed_original":83394573,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8384512,"data_stored":70434311,"data_compressed":1795097,"data_compressed_allocated":6225920,"data_compressed_original":69233432,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8331264,"data_stored":73488095,"data_compressed":2202008,"data_compressed_allocated":5746688,"data_compressed_original":71972677,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11378688,"data_stored":97726417,"data_compressed":2749656,"data_compressed_allocated":8658944,"data_compressed_original":96122692,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7036928,"data_stored":70308246,"data_compressed":1849516,"data_compressed_allocated":5320704,"data_compressed_original":69317527,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5763072,"data_stored":52830524,"data_compressed":1188881,"data_compressed_allocated":3936256,"data_compressed_original":51752827,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7847936,"data_stored":90274699,"data_compressed":1813038,"data_compressed_allocated":6144000,"data_compressed_original":89341302,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":2056974,"data_compressed":64441,"data_compressed_allocated":208896,"data_compressed_original":1875968,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:08:55.180 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T14:08:55.181 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:08:55.181 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6] 2024-09-16T14:08:55.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-16T14:08:55.241 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:55.236+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:55.242 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:55.242 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:08:55.242 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:55.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:55.341 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:55.334+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:55.345 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:08:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:08:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:08:55.345 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:55.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:55.443 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:55.438+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:55.451 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:08:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:08:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:08:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:08:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.451 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.930111136000001, 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.924839797000001, 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:08:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7423312830000004, 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:08:55.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161246 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4096000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.738939147, 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062177214000000001, 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161246 2024-09-16T14:08:55.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6199999999999999e-07 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710071+0000", 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.159e-06 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710074+0000", 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:08:55.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710076+0000", 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9439999999999999e-06 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772126+0000", 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062050007999999997 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772183+0000", 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6446999999999997e-05 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772247+0000", 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4399000000000005e-05 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.473 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7389307519999999, 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:08:55.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7348794730000003, 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:08:55.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7311265569999996, 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:08:55.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:08:55.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7270470289999995, 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:08:55.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:08:55.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7270304269999999, 2024-09-16T14:08:55.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:08:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7270144509999996, 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:08:55.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7269964990000002, 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:08:55.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7228013679999998, 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.509 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7227043899999996, 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:08:55.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7226798490000004, 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:08:55.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:08:55.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7191230869999998, 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:08:55.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:08:55.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:08:55.523 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7191053289999996, 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:08:55.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:08:55.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7190876660000001, 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:08:55.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:08:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7190704720000003, 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:08:55.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7153740539999998, 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:08:55.537 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:08:55.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.7030441019999998, 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.542 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:08:55.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:08:55.546 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:08:55.548 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:55.548 INFO:tasks.ceph.mon.a.smithi079.stderr:2024-09-16T14:08:55.464+0000 7f5ada529700 -1 mon.a@0(leader).osd e107 definitely_dead 0 2024-09-16T14:08:55.548 INFO:teuthology.orchestra.run.smithi079.stderr:marked out osd.3. 2024-09-16T14:08:55.549 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:08:55.550 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:55.542+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:55.648 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:55.644+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:55.657 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:55.658 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:55.658 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:55.658 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:55.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:55.749 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:55.742+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:55.765 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:55.766 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:55.774 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:55.850 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:55.848+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.535543298999997, 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:55.877 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:55.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:55.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:55.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:55.883 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.535520902000002, 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:55.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:55.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:55.890 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.534914190999999, 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:55.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.896 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.534839864000006, 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:55.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:55.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.563915072, 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:55.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:55.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.907 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.935956490999999, 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.908 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:55.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.405796025000001, 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.914 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:55.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:55.920 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.758247494000003, 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:55.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:55.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.211957253999998, 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.925 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:55.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:55.929 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.211940063, 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.930 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:55.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.934 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.211903046000003, 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.935 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:55.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:55.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:55.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.211882369999998, 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:55.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.941 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:55.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:55.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.211840189999997, 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:55.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:55.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.211805965000003, 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:55.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:55.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:55.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.954 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.211776249000003, 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.955 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:55.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.211756586999996, 2024-09-16T14:08:55.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:55.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.202996462000002, 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:55.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:55.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.968 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.118269920000003, 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.969 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:08:55.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.973 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.473266383, 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:08:55.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:08:55.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.473179522000001, 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:08:55.980 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:08:55.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:08:55.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.987 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.987 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.987 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:55.987 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:55.987 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:55.988 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:55.989 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:55.982+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:56.089 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:56.082+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:56.098 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:56.098 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:56.098 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:56.098 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:56.107 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:56.189 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:56.182+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:56.251 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:56.252 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:56.260 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:56.290 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:56.282+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:56.363 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.528985910000003, 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:56.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:08:56.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.528969570999998, 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:08:56.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.528944406000001, 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:08:56.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:08:56.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.528040439000002, 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.381 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:08:56.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:08:56.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.386 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.5280311, 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:08:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.527740977999997, 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.392 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:08:56.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.527522284, 2024-09-16T14:08:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:08:56.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:08:56.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.521128705000002, 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:56.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:08:56.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.408 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.517217142, 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:08:56.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:08:56.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.517185937000001, 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:56.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:08:56.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:08:56.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.419 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.516366497, 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:56.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.424 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.516321687999998, 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.425 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:08:56.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.513312433999999, 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:08:56.430 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.513295474000003, 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:56.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:08:56.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.513283977999997, 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.441 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:56.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:08:56.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.446 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.508537068000003, 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:56.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:08:56.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:08:56.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.500904437000003, 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:56.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:08:56.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.500892229999998, 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:08:56.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.498637315000003, 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.462 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:56.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:08:56.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.498413063999998, 2024-09-16T14:08:56.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:08:56.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:08:56.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.473 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.473 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.473 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.473 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:56.474 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:56.475 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:56.472+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:56.576 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:56.572+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:56.624 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:56.624 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:56.624 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:56.624 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:56.633 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:08:56.676 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:56.672+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:56.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:08:56.732 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:56.733 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:56.742 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:08:56.776 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:56.770+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:56.824 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.467318337999998, 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.463386919999998, 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.463362728, 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.459643456999999, 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.455726691999999, 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451344413000001, 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451320572, 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451305384000001, 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451290798000002, 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451187148000002, 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.892 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.451159422000003, 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:08:56.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:08:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.448485806000001, 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:08:56.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:08:56.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.448459092, 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.902 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.448418924999999, 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:08:56.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:08:56.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.448329209999997, 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.912 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:08:56.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.447920406000001, 2024-09-16T14:08:56.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:08:56.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:08:56.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.447820221999997, 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:08:56.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.925 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.446921584000002, 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:08:56.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.439722852000003, 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.930 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:08:56.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.437554560000002, 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:56.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:08:56.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:56.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:56.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.940 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:56.941 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:56.941 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:56.942 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:56.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:56.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:56.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:56.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:56.943 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:56.940+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:56.943 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:08:56.959 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:56.981 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:56.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:56.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:56.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:56.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:56.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:56.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:56.985 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:56.985 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:57.043 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:57.040+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:57.092 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:57.092 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:57.101 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:08:57.144 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:57.140+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:57.200 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:57.200 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:57.210 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:08:57.245 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:57.240+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:57.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:08:57.362 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:57.360+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:57.424 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:57.424 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:57.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:08:57.446 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:57.438+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:57.546 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:57.542+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:57.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:08:57.563 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:57.563 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:57.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:08:57.620 INFO:teuthology.orchestra.run.smithi079.stdout:flattening image image_client.0-clone41 2024-09-16T14:08:57.647 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:57.642+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:57.741 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:57.741 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:57.747 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:57.742+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:57.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:08:57.810 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-16T14:08:57.851 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:57.844+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:57.900 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:57.900 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:57.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:08:57.962 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:57.960+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:58.050 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:58.046+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:58.128 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:58.128 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:58.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:08:58.151 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:58.148+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:58.251 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:58.246+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:58.275 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:58.276 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:58.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:08:58.352 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:58.346+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:58.415 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:58.415 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:58.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:08:58.452 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:58.448+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:58.490 INFO:teuthology.orchestra.run.smithi079.stdout:4862 punch from 0xf9238c to 0xf9bf97, (0x9c0b bytes) 2024-09-16T14:08:58.521 INFO:teuthology.orchestra.run.smithi079.stdout:4863 write 0xdcfad25 thru 0xdd037aa (0x8a86 bytes) 2024-09-16T14:08:58.532 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:58.532 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:58.542 INFO:teuthology.orchestra.run.smithi079.stdout:4864 trunc from 0xdd037ab to 0x61a9c49 2024-09-16T14:08:58.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:08:58.551 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:08:58.563 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:58.560+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:58.654 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:08:58.652+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:08:58.668 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:58.668 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:58.679 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:08:58.755 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:58.750+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:58.785 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:58.785 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:58.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:08:58.857 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:58.852+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:08:58.905 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:58.906 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:58.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:08:58.958 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:58.950+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:58.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:08:59.016 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:59.016 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:59.027 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:08:59.059 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:08:59.056+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:08:59.131 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:08:59.136 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:08:59.136 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:08:59.146 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:08:59.151 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:08:59.159 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:59.154+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:59.171 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:08:59.191 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:08:59.212 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:08:59.241 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:08:59.246 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:59.246 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:59.246 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:59.246 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:59.247 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:59.255 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:08:59.260 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:08:59.254+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:08:59.264 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:08:59.288 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:08:59.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:08:59.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:08:59.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:08:59.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:08:59.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:08:59.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:08:59.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:08:59.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:08:59.357 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:59.357 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:59.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:59.357 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:59.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:59.357 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.01564294000001, 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:08:59.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:08:59.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:08:59.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.363 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.015620543, 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:08:59.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:08:59.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:08:59.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.01501383199999, 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:59.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:08:59.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.014939505, 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:08:59.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:08:59.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:08:59.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.044014712999996, 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:08:59.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.387 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.416056131999994, 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:08:59.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:08:59.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.394 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:08:59.394 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:59.394 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 87.885895665999996, 2024-09-16T14:08:59.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:08:59.394 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.394 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.394 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:08:59.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:08:59.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:08:59.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:08:59.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.238347134999998, 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101792 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.692056895, 2024-09-16T14:08:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019469391999999999, 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101792 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660485+0000", 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660487+0000", 2024-09-16T14:08:59.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0360000000000001e-06 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660494+0000", 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4989999999999999e-06 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660498+0000", 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9269999999999998e-06 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679779+0000", 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019281334000000001 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679865+0000", 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5470000000000007e-05 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679954+0000", 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9710000000000007e-05 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101793 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.692039704000003, 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019653470999999999, 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101793 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.411 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660502+0000", 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660503+0000", 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4299999999999998e-07 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660504+0000", 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8339999999999999e-06 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660529+0000", 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.442e-05 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660534+0000", 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0699999999999997e-06 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.679988+0000", 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019453841999999999 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680074+0000", 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6242999999999994e-05 2024-09-16T14:08:59.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680156+0000", 2024-09-16T14:08:59.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1619000000000006e-05 2024-09-16T14:08:59.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.415 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.692002686999999, 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:08:59.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.420 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.691982011, 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:08:59.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:08:59.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:59.425 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.691939830999999, 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:59.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:08:59.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.429 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.691905605999999, 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.430 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:08:59.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.691875889999999, 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:08:59.434 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:08:59.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.438 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.691856227999999, 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:08:59.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.683096102999997, 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:08:59.443 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:08:59.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.447 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.598369560999998, 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.448 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:08:59.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:08:59.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.953366024000001, 2024-09-16T14:08:59.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:08:59.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:08:59.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.953279163000001, 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.459 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:08:59.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:08:59.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.465 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:59.467 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:08:59.467 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:08:59.466+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:08:59.567 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:59.562+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:59.619 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:59.619 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:59.619 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:08:59.619 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:59.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:08:59.668 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:08:59.662+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:08:59.723 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:59.723 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:08:59.723 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:08:59.723 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:08:59.723 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:59.732 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:08:59.769 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:08:59.764+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:08:59.869 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:08:59.862+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.044374658000002, 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:08:59.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:08:59.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:08:59.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:08:59.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:08:59.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.044358318999997, 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.888 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:08:59.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.044333154, 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:08:59.893 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:08:59.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:08:59.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.898 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.043429187000001, 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:08:59.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:08:59.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:08:59.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.043419847999999, 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.904 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:08:59.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:08:59.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.909 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.043129725999997, 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:59.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:08:59.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.042911031999999, 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:08:59.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:08:59.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.036517453000002, 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:08:59.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:08:59.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:08:59.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.032605889999999, 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:08:59.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:08:59.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.931 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.032574685, 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:08:59.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:08:59.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.031755244999999, 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.937 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:08:59.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:08:59.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:59.942 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.031710435999997, 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:59.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:08:59.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.028701181999999, 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:08:59.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:08:59.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.028684222000003, 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:08:59.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:08:59.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.028672726000003, 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:08:59.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.963 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.023925816000002, 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:08:59.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.016293185000002, 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:59.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:08:59.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:59.974 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.016280977999998, 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:08:59.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.979 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.014026063000003, 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:08:59.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.013801811999997, 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:08:59.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:08:59.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:08:59.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.989 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:08:59.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:08:59.990 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:08:59.991 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:08:59.992 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:08:59.988+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:00.092 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:00.088+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:00.106 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3.ed243fb2 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:09:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.551164074, 2024-09-16T14:09:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.55120538299999999, 2024-09-16T14:09:00.106 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for peered", 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:09:00.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:09:00.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T14:09:00.110 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:00.115 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:00.193 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:00.186+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:00.213 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:00.213 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:00.222 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:00.294 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:00.290+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:00.322 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.944554881000002, 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:00.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:09:00.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.940623463000001, 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:00.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.940599270999996, 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:09:00.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:09:00.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:09:00.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.936880000000002, 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.337 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:09:00.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.932963235000003, 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:09:00.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:09:00.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.928580955999998, 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.346 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.928557114999997, 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:00.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.354 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.928541926999998, 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:00.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:09:00.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:09:00.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.928527340999999, 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:09:00.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:09:00.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.928423690999999, 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.363 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:09:00.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:09:00.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.928395965, 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.368 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:09:00.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.925722348999997, 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.372 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:09:00.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.925695634999997, 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.377 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:09:00.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.925655468000002, 2024-09-16T14:09:00.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:09:00.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.925565753000001, 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:09:00.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.925156948999998, 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:09:00.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:09:00.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.925056765000001, 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:09:00.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:09:00.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123990 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 12206080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.924158126999998, 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070506579, 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123990 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.400 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394849+0000", 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397824+0000", 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029752450000000001 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397828+0000", 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7519999999999998e-06 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397836+0000", 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-06 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.397841+0000", 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8620000000000002e-06 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464982+0000", 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067140191000000002 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465151+0000", 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016982299999999999 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.465355+0000", 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00020390600000000001 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.916959394999999, 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:09:00.404 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:09:00.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:09:00.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:09:00.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.914791102999999, 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:09:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:09:00.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:00.415 INFO:teuthology.orchestra.run.smithi079.stdout:4865 writesame 0xbb48fb thru 0xbbe598 data_size 0x1(0x9c9e bytes) 2024-09-16T14:09:00.415 INFO:teuthology.orchestra.run.smithi079.stdout:4866 write 0x5f4e429 thru 0x5f51bbb (0x3793 bytes) 2024-09-16T14:09:00.416 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:00.412+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:00.416 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:00.423 INFO:teuthology.orchestra.run.smithi079.stdout:4867 punch from 0x45ecd9e to 0x45ef358, (0x25ba bytes) 2024-09-16T14:09:00.423 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:00.423 INFO:teuthology.orchestra.run.smithi079.stdout:4869 read 0x1ec75db thru 0x1ecc87d (0x52a3 bytes) 2024-09-16T14:09:00.450 INFO:teuthology.orchestra.run.smithi079.stdout:4872 writesame 0xeb2f233 thru 0xeb36e25 data_size 0x7(0x7bf3 bytes) 2024-09-16T14:09:00.474 INFO:teuthology.orchestra.run.smithi079.stdout:4873 trunc from 0xeb36e26 to 0x254d94c 2024-09-16T14:09:00.516 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:00.510+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:00.526 INFO:teuthology.orchestra.run.smithi079.stdout:4874 write 0x45944ca thru 0x45a2edb (0xea12 bytes) 2024-09-16T14:09:00.536 INFO:teuthology.orchestra.run.smithi079.stdout:4876 read 0x1cefdc3 thru 0x1cfab2b (0xad69 bytes) 2024-09-16T14:09:00.543 INFO:teuthology.orchestra.run.smithi079.stdout:4877 trunc from 0x45a2edc to 0x26cd15f 2024-09-16T14:09:00.549 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6] out_osds: [7, 3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:09:00.550 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:09:00.550 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T14:09:00.550 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.942945 2024-09-16T14:09:00.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.9429450139845061 2024-09-16T14:09:00.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:09:00.573 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:00.573 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:00.573 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:00.573 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:00.583 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:00.585 INFO:teuthology.orchestra.run.smithi079.stdout:4879 writesame 0x2940f23 thru 0x29494fe data_size 0x1a(0x85dc bytes) 2024-09-16T14:09:00.589 INFO:teuthology.orchestra.run.smithi079.stdout:4880 writesame 0x1f42e59 thru 0x1f4c2dd data_size 0x1(0x9485 bytes) 2024-09-16T14:09:00.617 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:00.610+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:00.634 INFO:teuthology.orchestra.run.smithi079.stdout:4881 trunc from 0x29494ff to 0xbb63c6a 2024-09-16T14:09:00.645 INFO:teuthology.orchestra.run.smithi079.stdout:4884 trunc from 0xbb63c6a to 0xbe5474b 2024-09-16T14:09:00.666 INFO:teuthology.orchestra.run.smithi079.stdout:4886 trunc from 0xbe5474b to 0x419c60f 2024-09-16T14:09:00.678 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:00.678 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:00.678 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:00.687 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:00.699 INFO:teuthology.orchestra.run.smithi079.stdout:4887 write 0xc994c69 thru 0xc99dd13 (0x90ab bytes) 2024-09-16T14:09:00.704 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:00.704 INFO:teuthology.orchestra.run.smithi079.stdout:4890 write 0x5b944c6 thru 0x5b9bf72 (0x7aad bytes) 2024-09-16T14:09:00.710 INFO:teuthology.orchestra.run.smithi079.stdout:4891 read 0x5ff487 thru 0x60532d (0x5ea7 bytes) 2024-09-16T14:09:00.713 INFO:teuthology.orchestra.run.smithi079.stdout:4892 punch from 0x2038981 to 0x20427bb, (0x9e3a bytes) 2024-09-16T14:09:00.713 INFO:teuthology.orchestra.run.smithi079.stdout:4893 write 0x3ae0b8a thru 0x3ae8ec2 (0x8339 bytes) 2024-09-16T14:09:00.724 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:00.724 INFO:teuthology.orchestra.run.smithi079.stdout:4895 punch from 0x63a7bd1 to 0x63b0cec, (0x911b bytes) 2024-09-16T14:09:00.724 INFO:teuthology.orchestra.run.smithi079.stdout:4896 read 0x20965f6 thru 0x20a3b56 (0xd561 bytes) 2024-09-16T14:09:00.730 INFO:teuthology.orchestra.run.smithi079.stdout:4897 trunc from 0xc99dd14 to 0xd7a099f 2024-09-16T14:09:00.730 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:00.726+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:00.759 INFO:teuthology.orchestra.run.smithi079.stdout:4901 trunc from 0xd7a099f to 0x1fb78aa 2024-09-16T14:09:00.794 INFO:teuthology.orchestra.run.smithi079.stdout:4902 write 0x263af24 thru 0x263de78 (0x2f55 bytes) 2024-09-16T14:09:00.819 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:00.814+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:00.822 INFO:teuthology.orchestra.run.smithi079.stdout:4903 write 0x9168051 thru 0x9174498 (0xc448 bytes) 2024-09-16T14:09:00.829 INFO:teuthology.orchestra.run.smithi079.stdout:4905 read 0x298d2c4 thru 0x299b248 (0xdf85 bytes) 2024-09-16T14:09:00.834 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:00.834 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:00.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:00.834 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:00.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:00.834 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.421741209999993, 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:00.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:00.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.583778886999994, 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:00.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:00.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:00.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:00.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.056539899000001, 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:00.847 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:00.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:00.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.056516668, 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:00.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.056493721000002, 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:00.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:00.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.056477211999997, 2024-09-16T14:09:00.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:00.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.056459586000003, 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:00.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.869 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.050543842000003, 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:00.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:00.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.050526437000002, 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:00.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:00.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:00.878 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.050172070999999, 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:00.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:00.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:00.883 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.047028212000001, 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:00.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:00.888 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.047012926000001, 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:00.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:00.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.046996516999997, 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.893 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:00.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:00.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.046921320000003, 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:00.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:00.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.901 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.046823773, 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:00.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.046798531999997, 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:00.906 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:00.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:00.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.046769478000002, 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:00.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.043505488999998, 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:00.919 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.043487288000001, 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:00.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.04344811, 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:00.924 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:00.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:00.928 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:00.929 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:00.930 INFO:teuthology.orchestra.run.smithi079.stdout:4906 trunc from 0x9174499 to 0x10b252c 2024-09-16T14:09:00.930 INFO:teuthology.orchestra.run.smithi079.stdout:4909 writesame 0x65843f8 thru 0x658bd87 data_size 0x28(0x7990 bytes) 2024-09-16T14:09:00.930 INFO:teuthology.orchestra.run.smithi079.stdout:4910 write 0x9f88df7 thru 0x9f93ba1 (0xadab bytes) 2024-09-16T14:09:00.930 INFO:teuthology.orchestra.run.smithi079.stdout:4912 trunc from 0x9f93ba2 to 0x6f57b0c 2024-09-16T14:09:00.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:00.931 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:00.922+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:00.946 INFO:teuthology.orchestra.run.smithi079.stdout:4913 write 0x1a3884 thru 0x1a42b8 (0xa35 bytes) 2024-09-16T14:09:00.972 INFO:teuthology.orchestra.run.smithi079.stdout:4914 writesame 0x98bbfc thru 0x99af9b data_size 0x10(0xf3a0 bytes) 2024-09-16T14:09:00.980 INFO:teuthology.orchestra.run.smithi079.stdout:4915 trunc from 0x6f57b0c to 0x44b71c9 2024-09-16T14:09:00.992 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:00.992 INFO:teuthology.orchestra.run.smithi079.stdout:4917 read 0x120eb89 thru 0x1219790 (0xac08 bytes) 2024-09-16T14:09:00.992 INFO:teuthology.orchestra.run.smithi079.stdout:4918 read 0x24d7c07 thru 0x24e7ab0 (0xfeaa bytes) 2024-09-16T14:09:00.992 INFO:teuthology.orchestra.run.smithi079.stdout:4919 write 0x10260a3 thru 0x10294c8 (0x3426 bytes) 2024-09-16T14:09:00.996 INFO:teuthology.orchestra.run.smithi079.stdout:4920 punch from 0x25f3519 to 0x26022d8, (0xedbf bytes) 2024-09-16T14:09:00.996 INFO:teuthology.orchestra.run.smithi079.stdout:4922 writesame 0x12b29cb thru 0x12bf138 data_size 0x6(0xc76e bytes) 2024-09-16T14:09:01.029 INFO:teuthology.orchestra.run.smithi079.stdout:4924 write 0x60b87f0 thru 0x60c2e12 (0xa623 bytes) 2024-09-16T14:09:01.030 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:01.028+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:01.031 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:01.031 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:01.031 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:01.031 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:01.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:01.041 INFO:teuthology.orchestra.run.smithi079.stdout:4925 trunc from 0x60c2e13 to 0x3fe6f57 2024-09-16T14:09:01.063 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:01.063 INFO:teuthology.orchestra.run.smithi079.stdout:4928 read 0x14fca06 thru 0x14feb92 (0x218d bytes) 2024-09-16T14:09:01.090 INFO:teuthology.orchestra.run.smithi079.stdout:4929 write 0xb89a499 thru 0xb89d625 (0x318d bytes) 2024-09-16T14:09:01.098 INFO:teuthology.orchestra.run.smithi079.stdout:4930 write 0x915d2b4 thru 0x91660fb (0x8e48 bytes) 2024-09-16T14:09:01.104 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:01.104 INFO:teuthology.orchestra.run.smithi079.stdout:4932 punch from 0x3f2e416 to 0x3f34127, (0x5d11 bytes) 2024-09-16T14:09:01.104 INFO:teuthology.orchestra.run.smithi079.stdout:4933 write 0x326ffe9 thru 0x327cd62 (0xcd7a bytes) 2024-09-16T14:09:01.114 INFO:teuthology.orchestra.run.smithi079.stdout:4934 read 0x2205893 thru 0x220c951 (0x70bf bytes) 2024-09-16T14:09:01.114 INFO:teuthology.orchestra.run.smithi079.stdout:4935 write 0x44008bc thru 0x440d81d (0xcf62 bytes) 2024-09-16T14:09:01.126 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:01.131 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:01.126+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:01.132 INFO:teuthology.orchestra.run.smithi079.stdout:4937 trunc from 0xb89d626 to 0x782f94a 2024-09-16T14:09:01.149 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:01.149 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:01.150 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:01.157 INFO:teuthology.orchestra.run.smithi079.stdout:4938 read 0x3b4d2ea thru 0x3b55c3d (0x8954 bytes) 2024-09-16T14:09:01.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:01.171 INFO:teuthology.orchestra.run.smithi079.stdout:4939 write 0x8010410 thru 0x80150df (0x4cd0 bytes) 2024-09-16T14:09:01.179 INFO:teuthology.orchestra.run.smithi079.stdout:4940 writesame 0x60ce59c thru 0x60d1cd9 data_size 0x3(0x373e bytes) 2024-09-16T14:09:01.189 INFO:teuthology.orchestra.run.smithi079.stdout:4941 read 0x62ad9d4 thru 0x62aeb61 (0x118e bytes) 2024-09-16T14:09:01.190 INFO:teuthology.orchestra.run.smithi079.stdout:4942 write 0x22b2cdc thru 0x22b810e (0x5433 bytes) 2024-09-16T14:09:01.196 INFO:teuthology.orchestra.run.smithi079.stdout:4945 read 0x66dc8cf thru 0x66eba8c (0xf1be bytes) 2024-09-16T14:09:01.196 INFO:teuthology.orchestra.run.smithi079.stdout:4946 read 0x27a3dcf thru 0x27a5b33 (0x1d65 bytes) 2024-09-16T14:09:01.196 INFO:teuthology.orchestra.run.smithi079.stdout:4947 punch from 0xff2a27 to 0xffa8c2, (0x7e9b bytes) 2024-09-16T14:09:01.196 INFO:teuthology.orchestra.run.smithi079.stdout:4948 write 0x4659efc thru 0x4667037 (0xd13c bytes) 2024-09-16T14:09:01.205 INFO:teuthology.orchestra.run.smithi079.stdout:4949 write 0x21a59c6 thru 0x21ab2a9 (0x58e4 bytes) 2024-09-16T14:09:01.212 INFO:teuthology.orchestra.run.smithi079.stdout:4950 read 0x1ac5c46 thru 0x1ad02b4 (0xa66f bytes) 2024-09-16T14:09:01.213 INFO:teuthology.orchestra.run.smithi079.stdout:4952 read 0x11720aa thru 0x117c335 (0xa28c bytes) 2024-09-16T14:09:01.226 INFO:teuthology.orchestra.run.smithi079.stdout:4955 trunc from 0x80150e0 to 0x23a80dd 2024-09-16T14:09:01.232 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:01.226+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:01.255 INFO:teuthology.orchestra.run.smithi079.stdout:4956 read 0x1fdcb43 thru 0x1fe9e14 (0xd2d2 bytes) 2024-09-16T14:09:01.255 INFO:teuthology.orchestra.run.smithi079.stdout:4957 write 0x21e5cab thru 0x21eead6 (0x8e2c bytes) 2024-09-16T14:09:01.258 INFO:teuthology.orchestra.run.smithi079.stdout:4958 read 0x1bc8ed3 thru 0x1bceef4 (0x6022 bytes) 2024-09-16T14:09:01.278 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:01.278 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 34.038190180999997, 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:01.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.678037138000001, 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.678028139, 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.678019788, 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677987104, 2024-09-16T14:09:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677977866999999, 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677969335, 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677960757999999, 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677951396999999, 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677943285, 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677931581999999, 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677922969000001, 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677914863, 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677906521000001, 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677897982999999, 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677889650999999, 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677881408999999, 2024-09-16T14:09:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677871846, 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677862465, 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.677852824, 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:01.373 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:01.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:01.374 INFO:teuthology.orchestra.run.smithi079.stdout:4959 writesame 0x75bdbf2 thru 0x75c1842 data_size 0x1(0x3c51 bytes) 2024-09-16T14:09:01.375 INFO:teuthology.orchestra.run.smithi079.stdout:4961 read 0x31f4ab3 thru 0x31f612f (0x167d bytes) 2024-09-16T14:09:01.375 INFO:teuthology.orchestra.run.smithi079.stdout:4963 read 0x6a00329 thru 0x6a088cb (0x85a3 bytes) 2024-09-16T14:09:01.375 INFO:teuthology.orchestra.run.smithi079.stdout:4965 write 0x16a8aef thru 0x16b6d21 (0xe233 bytes) 2024-09-16T14:09:01.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:09:01.376 INFO:teuthology.orchestra.run.smithi079.stdout:4966 trunc from 0x75c1843 to 0x7481519 2024-09-16T14:09:01.377 INFO:teuthology.orchestra.run.smithi079.stdout:4967 trunc from 0x7481519 to 0x1989e8b 2024-09-16T14:09:01.377 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:01.377 INFO:teuthology.orchestra.run.smithi079.stdout:4969 read 0x156c599 thru 0x156e01c (0x1a84 bytes) 2024-09-16T14:09:01.377 INFO:teuthology.orchestra.run.smithi079.stdout:4970 trunc from 0x1989e8b to 0x9be06a8 2024-09-16T14:09:01.377 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:09:01.378 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.4 primary-affinity to 0.942945 (8617962) 2024-09-16T14:09:01.378 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:01.372+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:01.378 INFO:teuthology.orchestra.run.smithi079.stdout:4971 read 0x7ff3e70 thru 0x7ffa827 (0x69b8 bytes) 2024-09-16T14:09:01.391 INFO:teuthology.orchestra.run.smithi079.stdout:4972 trunc from 0x9be06a8 to 0x195c624 2024-09-16T14:09:01.469 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:01.470 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:01.475 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:01.472+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:01.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:09:01.516 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:09:01.550 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:01.576 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:01.570+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:01.578 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:01.589 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:01.589 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:01.594 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:01.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:09:01.620 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:01.640 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:01.663 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:01.677 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:01.670+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:01.685 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:01.687 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:01.688 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:01.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:01.778 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:01.770+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:01.791 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:01.792 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:01.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:01.879 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:01.876+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:01.901 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:01.901 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:01.910 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:01.979 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:01.976+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:02.006 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.006 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:02.080 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:02.074+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:02.113 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.113 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:02.180 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:02.178+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:02.220 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.221 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:02.281 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:02.274+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:02.329 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.330 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:02.382 INFO:teuthology.orchestra.run.smithi079.stdout:4974 write 0x149fc77 thru 0x14ae8fb (0xec85 bytes) 2024-09-16T14:09:02.382 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:02.380+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:02.410 INFO:teuthology.orchestra.run.smithi079.stdout:4975 trunc from 0x195c624 to 0xe0baa41 2024-09-16T14:09:02.422 INFO:teuthology.orchestra.run.smithi079.stdout:4976 read 0x5984987 thru 0x598f2c0 (0xa93a bytes) 2024-09-16T14:09:02.422 INFO:teuthology.orchestra.run.smithi079.stdout:4977 read 0x9415909 thru 0x94226e9 (0xcde1 bytes) 2024-09-16T14:09:02.422 INFO:teuthology.orchestra.run.smithi079.stdout:4978 punch from 0x514d1b4 to 0x515997c, (0xc7c8 bytes) 2024-09-16T14:09:02.422 INFO:teuthology.orchestra.run.smithi079.stdout:4979 read 0x34d8ca8 thru 0x34e2076 (0x93cf bytes) 2024-09-16T14:09:02.435 INFO:teuthology.orchestra.run.smithi079.stdout:4980 write 0xedb992a thru 0xedbb6cf (0x1da6 bytes) 2024-09-16T14:09:02.441 INFO:teuthology.orchestra.run.smithi079.stdout:4981 read 0xab87fbf thru 0xab8a0b6 (0x20f8 bytes) 2024-09-16T14:09:02.452 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.453 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.456 INFO:teuthology.orchestra.run.smithi079.stdout:4983 trunc from 0xedbb6d0 to 0x79f1c71 2024-09-16T14:09:02.463 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:02.483 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:02.480+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:02.506 INFO:teuthology.orchestra.run.smithi079.stdout:4984 write 0xd334fb6 thru 0xd340276 (0xb2c1 bytes) 2024-09-16T14:09:02.514 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:02.514 INFO:teuthology.orchestra.run.smithi079.stdout:4987 write 0xcb35dcc thru 0xcb39fb6 (0x41eb bytes) 2024-09-16T14:09:02.521 INFO:teuthology.orchestra.run.smithi079.stdout:4988 read 0x781eb6f thru 0x782976f (0xac01 bytes) 2024-09-16T14:09:02.521 INFO:teuthology.orchestra.run.smithi079.stdout:4989 read 0x8cdcd9b thru 0x8ce6071 (0x92d7 bytes) 2024-09-16T14:09:02.521 INFO:teuthology.orchestra.run.smithi079.stdout:4990 punch from 0x6371950 to 0x637af2c, (0x95dc bytes) 2024-09-16T14:09:02.521 INFO:teuthology.orchestra.run.smithi079.stdout:4991 writesame 0x403a2b0 thru 0x403df4a data_size 0x1d(0x3c9b bytes) 2024-09-16T14:09:02.530 INFO:teuthology.orchestra.run.smithi079.stdout:4992 read 0x266bf2b thru 0x266d9ea (0x1ac0 bytes) 2024-09-16T14:09:02.530 INFO:teuthology.orchestra.run.smithi079.stdout:4993 read 0x53bfd42 thru 0x53cc9c0 (0xcc7f bytes) 2024-09-16T14:09:02.531 INFO:teuthology.orchestra.run.smithi079.stdout:4994 read 0x132301e thru 0x1332cc4 (0xfca7 bytes) 2024-09-16T14:09:02.531 INFO:teuthology.orchestra.run.smithi079.stdout:4996 write 0x71caa41 thru 0x71cab02 (0xc2 bytes) 2024-09-16T14:09:02.539 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:02.539 INFO:teuthology.orchestra.run.smithi079.stdout:4998 writesame 0x1936de9 thru 0x1943957 data_size 0x1(0xcb6f bytes) 2024-09-16T14:09:02.557 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.557 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.566 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:02.582 INFO:teuthology.orchestra.run.smithi079.stdout:4999 trunc from 0xd340277 to 0xe69d19e 2024-09-16T14:09:02.584 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:02.578+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:02.594 INFO:teuthology.orchestra.run.smithi079.stdout:5000 trunc from 0xe69d19e to 0xe3d25aa 2024-09-16T14:09:02.606 INFO:teuthology.orchestra.run.smithi079.stdout:5001 read 0x8637af2 thru 0x863ffce (0x84dd bytes) 2024-09-16T14:09:02.613 INFO:teuthology.orchestra.run.smithi079.stdout:5003 trunc from 0xe3d25aa to 0xbcde973 2024-09-16T14:09:02.637 INFO:teuthology.orchestra.run.smithi079.stdout:5004 writesame 0x4c74028 thru 0x4c7ed3f data_size 0x8(0xad18 bytes) 2024-09-16T14:09:02.648 INFO:teuthology.orchestra.run.smithi079.stdout:5005 read 0x564ba4f thru 0x5651252 (0x5804 bytes) 2024-09-16T14:09:02.648 INFO:teuthology.orchestra.run.smithi079.stdout:5006 write 0xae5b69e thru 0xae5d6af (0x2012 bytes) 2024-09-16T14:09:02.669 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.669 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.672 INFO:teuthology.orchestra.run.smithi079.stdout:5007 writesame 0xccfef4e thru 0xcd0cf63 data_size 0x9(0xe016 bytes) 2024-09-16T14:09:02.673 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:02.673 INFO:teuthology.orchestra.run.smithi079.stdout:5009 read 0x28780b7 thru 0x2884f9c (0xcee6 bytes) 2024-09-16T14:09:02.673 INFO:teuthology.orchestra.run.smithi079.stdout:5010 write 0xbcb3f7a thru 0xbcbba06 (0x7a8d bytes) 2024-09-16T14:09:02.678 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:02.681 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:02.698 INFO:teuthology.orchestra.run.smithi079.stdout:5012 write 0xe5942c8 thru 0xe59f4b6 (0xb1ef bytes) 2024-09-16T14:09:02.716 INFO:teuthology.orchestra.run.smithi079.stdout:5014 trunc from 0xe59f4b7 to 0xa1d8634 2024-09-16T14:09:02.722 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:02.718+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:02.757 INFO:teuthology.orchestra.run.smithi079.stdout:5015 write 0x26f6c8b thru 0x26f9f11 (0x3287 bytes) 2024-09-16T14:09:02.765 INFO:teuthology.orchestra.run.smithi079.stdout:5017 writesame 0x414b514 thru 0x414c9ec data_size 0x3(0x14d9 bytes) 2024-09-16T14:09:02.770 INFO:teuthology.orchestra.run.smithi079.stdout:5020 write 0x65f2ac5 thru 0x65f6235 (0x3771 bytes) 2024-09-16T14:09:02.778 INFO:teuthology.orchestra.run.smithi079.stdout:5021 punch from 0xa5e52 to 0xac2a7, (0x6455 bytes) 2024-09-16T14:09:02.778 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.778 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.786 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:02.784+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:02.787 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:02.790 INFO:teuthology.orchestra.run.smithi079.stdout:5023 trunc from 0xa1d8634 to 0x47f2992 2024-09-16T14:09:02.822 INFO:teuthology.orchestra.run.smithi079.stdout:5024 punch from 0x3baae7d to 0x3bab1db, (0x35e bytes) 2024-09-16T14:09:02.849 INFO:teuthology.orchestra.run.smithi079.stdout:5025 write 0x9ea52d5 thru 0x9eb0997 (0xb6c3 bytes) 2024-09-16T14:09:02.877 INFO:teuthology.orchestra.run.smithi079.stdout:5026 write 0xda2c2b7 thru 0xda2fba2 (0x38ec bytes) 2024-09-16T14:09:02.883 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.883 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:02.883 INFO:teuthology.orchestra.run.smithi079.stdout:5027 punch from 0xaa091c0 to 0xaa12db7, (0x9bf7 bytes) 2024-09-16T14:09:02.883 INFO:teuthology.orchestra.run.smithi079.stdout:5029 read 0xa181893 thru 0xa185b5f (0x42cd bytes) 2024-09-16T14:09:02.883 INFO:teuthology.orchestra.run.smithi079.stdout:5030 write 0xa5ad776 thru 0xa5b1b3c (0x43c7 bytes) 2024-09-16T14:09:02.887 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:02.884+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:02.888 INFO:teuthology.orchestra.run.smithi079.stdout:5032 read 0x403132c thru 0x403aab6 (0x978b bytes) 2024-09-16T14:09:02.890 INFO:teuthology.orchestra.run.smithi079.stdout:5034 read 0x88cf9b3 thru 0x88dc21c (0xc86a bytes) 2024-09-16T14:09:02.890 INFO:teuthology.orchestra.run.smithi079.stdout:5037 punch from 0x84bf6c8 to 0x84c522a, (0x5b62 bytes) 2024-09-16T14:09:02.890 INFO:teuthology.orchestra.run.smithi079.stdout:5040 read 0xb49f2fc thru 0xb4a0e97 (0x1b9c bytes) 2024-09-16T14:09:02.890 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:02.890 INFO:teuthology.orchestra.run.smithi079.stdout:5043 read 0x9e36363 thru 0x9e45028 (0xecc6 bytes) 2024-09-16T14:09:02.890 INFO:teuthology.orchestra.run.smithi079.stdout:5045 read 0x37b100e thru 0x37b5404 (0x43f7 bytes) 2024-09-16T14:09:02.891 INFO:teuthology.orchestra.run.smithi079.stdout:5046 writesame 0x6d44d09 thru 0x6d4d179 data_size 0x1(0x8471 bytes) 2024-09-16T14:09:02.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:02.923 INFO:teuthology.orchestra.run.smithi079.stdout:5047 trunc from 0xda2fba3 to 0xccf07b7 2024-09-16T14:09:02.945 INFO:teuthology.orchestra.run.smithi079.stdout:5048 write 0x978d726 thru 0x978d779 (0x54 bytes) 2024-09-16T14:09:02.948 INFO:teuthology.orchestra.run.smithi079.stdout:5050 write 0x8150c2e thru 0x81580f9 (0x74cc bytes) 2024-09-16T14:09:02.955 INFO:teuthology.orchestra.run.smithi079.stdout:5053 punch from 0x6c43ab8 to 0x6c48ef0, (0x5438 bytes) 2024-09-16T14:09:02.961 INFO:teuthology.orchestra.run.smithi079.stdout:5054 trunc from 0xccf07b7 to 0xb47a413 2024-09-16T14:09:02.972 INFO:teuthology.orchestra.run.smithi079.stdout:5055 writesame 0x4e38698 thru 0x4e3b290 data_size 0x1(0x2bf9 bytes) 2024-09-16T14:09:02.988 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:02.984+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:02.988 INFO:teuthology.orchestra.run.smithi079.stdout:5056 read 0xaac09eb thru 0xaac5bf7 (0x520d bytes) 2024-09-16T14:09:02.988 INFO:teuthology.orchestra.run.smithi079.stdout:5057 read 0x97570f2 thru 0x975e385 (0x7294 bytes) 2024-09-16T14:09:02.989 INFO:teuthology.orchestra.run.smithi079.stdout:5058 write 0x4e5cd3 thru 0x4ee350 (0x867e bytes) 2024-09-16T14:09:02.997 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:02.998 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.008 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout:5059 writesame 0xcd4dcf1 thru 0xcd56a16 data_size 0x59(0x8d26 bytes) 2024-09-16T14:09:03.009 INFO:teuthology.orchestra.run.smithi079.stdout:5060 punch from 0x66c56a1 to 0x66cf6cb, (0xa02a bytes) 2024-09-16T14:09:03.016 INFO:teuthology.orchestra.run.smithi079.stdout:5061 trunc from 0xcd56a17 to 0x9cc2cfe 2024-09-16T14:09:03.042 INFO:teuthology.orchestra.run.smithi079.stdout:5062 read 0x96f387c thru 0x96f5980 (0x2105 bytes) 2024-09-16T14:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout:5064 read 0x3afbc73 thru 0x3b0b244 (0xf5d2 bytes) 2024-09-16T14:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:03.043 INFO:teuthology.orchestra.run.smithi079.stdout:5067 write 0x74da265 thru 0x74e7d7a (0xdb16 bytes) 2024-09-16T14:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout:5068 punch from 0x817c964 to 0x8188bc8, (0xc264 bytes) 2024-09-16T14:09:03.051 INFO:teuthology.orchestra.run.smithi079.stdout:5069 writesame 0x480d760 thru 0x4812e22 data_size 0x13(0x56c3 bytes) 2024-09-16T14:09:03.073 INFO:teuthology.orchestra.run.smithi079.stdout:5070 trunc from 0x9cc2cfe to 0x33082b0 2024-09-16T14:09:03.088 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:03.084+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:03.099 INFO:teuthology.orchestra.run.smithi079.stdout:5071 punch from 0x18e314d to 0x18f0f34, (0xdde7 bytes) 2024-09-16T14:09:03.108 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:03.108 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.116 INFO:teuthology.orchestra.run.smithi079.stdout:5073 trunc from 0x33082b0 to 0xa694895 2024-09-16T14:09:03.117 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:03.139 INFO:teuthology.orchestra.run.smithi079.stdout:5076 writesame 0xa771f06 thru 0xa7795ee data_size 0x49(0x76e9 bytes) 2024-09-16T14:09:03.139 INFO:teuthology.orchestra.run.smithi079.stdout:5077 clone 43 order 22 su 524288 sc 11 2024-09-16T14:09:03.189 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:03.184+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:03.209 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:03.209 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:03.290 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:03.282+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:03.317 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:03.317 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.325 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:03.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:09:03.423 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:03.423 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.431 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:03.428+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:03.432 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:03.492 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:03.486+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:03.527 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:03.528 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.537 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:03.593 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:03.586+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:03.641 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:03.641 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:03.694 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone42 from 0xa7795ef (overlap 0x10b252c) to 0x859296 2024-09-16T14:09:03.694 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:03.692+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:03.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:03.771 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:03.771 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:03.794 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:03.790+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:03.817 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-16T14:09:03.841 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:09:03.860 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:03.884 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:03.885 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:03.885 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:03.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:03.896 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:03.890+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:03.904 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:03.925 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:03.949 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:03.971 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:03.993 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.752969174999997, 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392816132, 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392807133, 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392798782, 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392766097999999, 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392756861000001, 2024-09-16T14:09:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392748329, 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392739752000001, 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392730391000001, 2024-09-16T14:09:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392722279000001, 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392710576000001, 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392701963, 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392693856999999, 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392685515, 2024-09-16T14:09:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392676977000001, 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392668645000001, 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392660403000001, 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.39265084, 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392641459, 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.392631818, 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.087 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:04.088 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:09:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:04.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:09:04.091 INFO:teuthology.orchestra.run.smithi079.stdout:5078 read 0xa421c9d thru 0xa423ef7 (0x225b bytes) 2024-09-16T14:09:04.092 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.092 INFO:teuthology.orchestra.run.smithi079.stdout:5080 read 0xee35cc thru 0xeea1ce (0x6c03 bytes) 2024-09-16T14:09:04.092 INFO:teuthology.orchestra.run.smithi079.stdout:5082 write 0x4218c7c thru 0x421bbe6 (0x2f6b bytes) 2024-09-16T14:09:04.092 INFO:teuthology.orchestra.run.smithi079.stdout:5083 punch from 0xa1c1307 to 0xa1c8954, (0x764d bytes) 2024-09-16T14:09:04.092 INFO:teuthology.orchestra.run.smithi079.stdout:5084 read 0x53d7a50 thru 0x53e21fa (0xa7ab bytes) 2024-09-16T14:09:04.092 INFO:teuthology.orchestra.run.smithi079.stdout:5085 write 0x4e6000d thru 0x4e63278 (0x326c bytes) 2024-09-16T14:09:04.092 INFO:teuthology.orchestra.run.smithi079.stdout:5086 trunc from 0xa7795ef to 0xada428f 2024-09-16T14:09:04.092 INFO:teuthology.orchestra.run.smithi079.stdout:5087 write 0x1143550 thru 0x11470b0 (0x3b61 bytes) 2024-09-16T14:09:04.092 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:04.082+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:04.112 INFO:teuthology.orchestra.run.smithi079.stdout:5088 punch from 0x4cb7884 to 0x4cbd20d, (0x5989 bytes) 2024-09-16T14:09:04.112 INFO:teuthology.orchestra.run.smithi079.stdout:5089 read 0x83d36cf thru 0x83de1b5 (0xaae7 bytes) 2024-09-16T14:09:04.112 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.127 INFO:teuthology.orchestra.run.smithi079.stdout:5092 write 0xb8f85dc thru 0xb904ca0 (0xc6c5 bytes) 2024-09-16T14:09:04.149 INFO:teuthology.orchestra.run.smithi079.stdout:5093 write 0xe13a1e7 thru 0xe1472b5 (0xd0cf bytes) 2024-09-16T14:09:04.156 INFO:teuthology.orchestra.run.smithi079.stdout:5094 punch from 0x1b39adb to 0x1b3a59a, (0xabf bytes) 2024-09-16T14:09:04.156 INFO:teuthology.orchestra.run.smithi079.stdout:5095 read 0xd125e80 thru 0xd12868b (0x280c bytes) 2024-09-16T14:09:04.156 INFO:teuthology.orchestra.run.smithi079.stdout:5096 write 0xcf14941 thru 0xcf24509 (0xfbc9 bytes) 2024-09-16T14:09:04.162 INFO:teuthology.orchestra.run.smithi079.stdout:5097 trunc from 0xe1472b6 to 0xbc48d92 2024-09-16T14:09:04.183 INFO:teuthology.orchestra.run.smithi079.stdout:5098 write 0x74a68be thru 0x74a88c2 (0x2005 bytes) 2024-09-16T14:09:04.189 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.189 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:04.182+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:04.190 INFO:teuthology.orchestra.run.smithi079.stdout:5100 trunc from 0xbc48d92 to 0xb2c16e1 2024-09-16T14:09:04.200 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:183600 3.13 3:cac04a1c:::rbd_object_map.1136b1466cf:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T14:09:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:04.197308+0000", 2024-09-16T14:09:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0019988639999999999, 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002050807, 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 183600 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197308+0000", 2024-09-16T14:09:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197308+0000", 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197310+0000", 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7719999999999999e-06 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197317+0000", 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2629999999999997e-06 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197334+0000", 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7249999999999999e-05 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197345+0000", 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1029000000000001e-05 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197415+0000", 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9469999999999997e-05 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197518+0000", 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102843 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197919+0000", 2024-09-16T14:09:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000401118 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:183601 3.13 3:cac04a1c:::rbd_object_map.1136b1466cf:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:04.197424+0000", 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.001882416, 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020934009999999999, 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 183601 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197424+0000", 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197424+0000", 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197426+0000", 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.68e-06 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197432+0000", 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2990000000000002e-06 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197446+0000", 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3658999999999999e-05 2024-09-16T14:09:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.197453+0000", 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4479999999999997e-06 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.198801+0000", 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013471100000000001 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:04.198916+0000", 2024-09-16T14:09:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115664 2024-09-16T14:09:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 2 2024-09-16T14:09:04.209 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:04.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:09:04.210 INFO:teuthology.orchestra.run.smithi079.stdout:5101 write 0x5e110c0 thru 0x5e132c2 (0x2203 bytes) 2024-09-16T14:09:04.216 INFO:teuthology.orchestra.run.smithi079.stdout:5102 read 0xa5f60e8 thru 0xa5f720d (0x1126 bytes) 2024-09-16T14:09:04.217 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.217 INFO:teuthology.orchestra.run.smithi079.stdout:5104 read 0x5799a05 thru 0x57a5add (0xc0d9 bytes) 2024-09-16T14:09:04.226 INFO:teuthology.orchestra.run.smithi079.stdout:5105 trunc from 0xb2c16e1 to 0x611babf 2024-09-16T14:09:04.273 INFO:teuthology.orchestra.run.smithi079.stdout:5106 write 0x8cac2e5 thru 0x8cacaca (0x7e6 bytes) 2024-09-16T14:09:04.290 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:04.288+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:04.302 INFO:teuthology.orchestra.run.smithi079.stdout:5107 write 0xe620e48 thru 0xe626c01 (0x5dba bytes) 2024-09-16T14:09:04.311 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.311 INFO:teuthology.orchestra.run.smithi079.stdout:5109 read 0x586ab73 thru 0x5875d8b (0xb219 bytes) 2024-09-16T14:09:04.311 INFO:teuthology.orchestra.run.smithi079.stdout:5111 write 0x2e5675 thru 0x2efa0c (0xa398 bytes) 2024-09-16T14:09:04.321 INFO:teuthology.orchestra.run.smithi079.stdout:5113 write 0x83f54c9 thru 0x83fce14 (0x794c bytes) 2024-09-16T14:09:04.324 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:04.324 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:04.324 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:04.324 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:04.325 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:04.325 INFO:teuthology.orchestra.run.smithi079.stdout:5114 read 0xa590362 thru 0xa599acc (0x976b bytes) 2024-09-16T14:09:04.325 INFO:teuthology.orchestra.run.smithi079.stdout:5115 read 0x91b890f thru 0x91b8ecc (0x5be bytes) 2024-09-16T14:09:04.325 INFO:teuthology.orchestra.run.smithi079.stdout:5116 write 0x36eec48 thru 0x36f07bc (0x1b75 bytes) 2024-09-16T14:09:04.327 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.328 INFO:teuthology.orchestra.run.smithi079.stdout:5118 punch from 0xc3ef564 to 0xc3faa9f, (0xb53b bytes) 2024-09-16T14:09:04.328 INFO:teuthology.orchestra.run.smithi079.stdout:5119 read 0x3c96828 thru 0x3c99e8e (0x3667 bytes) 2024-09-16T14:09:04.328 INFO:teuthology.orchestra.run.smithi079.stdout:5120 read 0x623d394 thru 0x623fe99 (0x2b06 bytes) 2024-09-16T14:09:04.329 INFO:teuthology.orchestra.run.smithi079.stdout:5122 read 0x957667 thru 0x95dcf2 (0x668c bytes) 2024-09-16T14:09:04.331 INFO:teuthology.orchestra.run.smithi079.stdout:5123 read 0xc832003 thru 0xc840aed (0xeaeb bytes) 2024-09-16T14:09:04.333 INFO:teuthology.orchestra.run.smithi079.stdout:5126 trunc from 0xe626c02 to 0xcf47d09 2024-09-16T14:09:04.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:09:04.354 INFO:teuthology.orchestra.run.smithi079.stdout:5127 read 0xcba533a thru 0xcbabffb (0x6cc2 bytes) 2024-09-16T14:09:04.354 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.354 INFO:teuthology.orchestra.run.smithi079.stdout:5131 punch from 0x9a7ffa5 to 0x9a83f84, (0x3fdf bytes) 2024-09-16T14:09:04.354 INFO:teuthology.orchestra.run.smithi079.stdout:5133 read 0x11c8722 thru 0x11d0102 (0x79e1 bytes) 2024-09-16T14:09:04.355 INFO:teuthology.orchestra.run.smithi079.stdout:5134 read 0x95f92ee thru 0x95fb702 (0x2415 bytes) 2024-09-16T14:09:04.355 INFO:teuthology.orchestra.run.smithi079.stdout:5135 write 0x40eb038 thru 0x40eb3b8 (0x381 bytes) 2024-09-16T14:09:04.362 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.365 INFO:teuthology.orchestra.run.smithi079.stdout:5137 trunc from 0xcf47d09 to 0xb656f33 2024-09-16T14:09:04.375 INFO:teuthology.orchestra.run.smithi079.stdout:5138 write 0x22fe6c0 thru 0x230a1c1 (0xbb02 bytes) 2024-09-16T14:09:04.393 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:04.388+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:04.405 INFO:teuthology.orchestra.run.smithi079.stdout:5139 write 0xc58e7fc thru 0xc598a06 (0xa20b bytes) 2024-09-16T14:09:04.429 INFO:teuthology.orchestra.run.smithi079.stdout:5141 trunc from 0xc598a07 to 0x2989013 2024-09-16T14:09:04.469 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.470 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556514958999998, 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:09:04.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:09:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:09:04.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:09:04.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.476 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556505232000006, 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:09:04.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:09:04.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:09:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:09:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556481668999993, 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:09:04.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:09:04.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:09:04.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556454430000002, 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.489 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:09:04.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:09:04.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556445017000001, 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:09:04.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:09:04.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556420697999997, 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:09:04.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:09:04.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:09:04.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.505 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556401092000002, 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:09:04.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:09:04.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.511 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556380903999994, 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:09:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556357634999998, 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:09:04.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556327385000003, 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:09:04.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:09:04.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:09:04.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:09:04.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:09:04.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:09:04.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:09:04.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:09:04.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:09:04.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:09:04.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556310632999995, 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:09:04.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:09:04.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:09:04.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:09:04.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:09:04.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:09:04.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:09:04.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:09:04.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:09:04.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:09:04.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:09:04.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:09:04.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:09:04.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:09:04.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.538 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.538 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:09:04.538 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556293369000002, 2024-09-16T14:09:04.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:09:04.539 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.539 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.539 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556254533000001, 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:09:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:09:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556219705000004, 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556183818999997, 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:09:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556161227000004, 2024-09-16T14:09:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:09:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.566 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556132155, 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:09:04.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556113808999996, 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.572 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:09:04.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:04.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.55609647, 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.578 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:09:04.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:09:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:09:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:09:04.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:09:04.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:09:04.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:09:04.584 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.556080546999993, 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:09:04.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:09:04.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:09:04.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:09:04.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.589 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:04.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:04.592 INFO:teuthology.orchestra.run.smithi079.stdout:5144 write 0xa012d96 thru 0xa0147c5 (0x1a30 bytes) 2024-09-16T14:09:04.592 INFO:teuthology.orchestra.run.smithi079.stdout:5145 trunc from 0xa0147c6 to 0x23b5f1a 2024-09-16T14:09:04.592 INFO:teuthology.orchestra.run.smithi079.stdout:5146 write 0x7fa0f3 thru 0x7fe1f5 (0x4103 bytes) 2024-09-16T14:09:04.592 INFO:teuthology.orchestra.run.smithi079.stdout:5147 read 0x168085b thru 0x168bae4 (0xb28a bytes) 2024-09-16T14:09:04.592 INFO:teuthology.orchestra.run.smithi079.stdout:5148 trunc from 0x23b5f1a to 0x6615c51 2024-09-16T14:09:04.592 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:09:04.593 INFO:teuthology.orchestra.run.smithi079.stdout:5153 write 0x84966ab thru 0x84a1f94 (0xb8ea bytes) 2024-09-16T14:09:04.593 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:04.585+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:04.595 INFO:teuthology.orchestra.run.smithi079.stdout:5155 read 0x187ba20 thru 0x188a45f (0xea40 bytes) 2024-09-16T14:09:04.597 INFO:teuthology.orchestra.run.smithi079.stdout:5156 punch from 0xd460ed to 0xd54986, (0xe899 bytes) 2024-09-16T14:09:04.597 INFO:teuthology.orchestra.run.smithi079.stdout:5157 write 0x7d2392d thru 0x7d25360 (0x1a34 bytes) 2024-09-16T14:09:04.607 INFO:teuthology.orchestra.run.smithi079.stdout:5158 write 0x8274289 thru 0x82818d2 (0xd64a bytes) 2024-09-16T14:09:04.613 INFO:teuthology.orchestra.run.smithi079.stdout:5160 writesame 0x71a6ce3 thru 0x71b18b0 data_size 0x1(0xabce bytes) 2024-09-16T14:09:04.641 INFO:teuthology.orchestra.run.smithi079.stdout:5161 write 0x1d48bc3 thru 0x1d4dd91 (0x51cf bytes) 2024-09-16T14:09:04.645 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.645 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.646 INFO:teuthology.orchestra.run.smithi079.stdout:5164 read 0x3c0092f thru 0x3c0bfcb (0xb69d bytes) 2024-09-16T14:09:04.646 INFO:teuthology.orchestra.run.smithi079.stdout:5165 punch from 0x684b4e2 to 0x68522f5, (0x6e13 bytes) 2024-09-16T14:09:04.646 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.646 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.646 INFO:teuthology.orchestra.run.smithi079.stdout:5168 read 0x224d35d thru 0x224eac6 (0x176a bytes) 2024-09-16T14:09:04.647 INFO:teuthology.orchestra.run.smithi079.stdout:5169 writesame 0x1c5e977 thru 0x1c6b51e data_size 0x13(0xcba8 bytes) 2024-09-16T14:09:04.651 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.651 INFO:teuthology.orchestra.run.smithi079.stdout:5171 read 0x584acce thru 0x5857f6c (0xd29f bytes) 2024-09-16T14:09:04.651 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.651 INFO:teuthology.orchestra.run.smithi079.stdout:5174 punch from 0x2ee0af2 to 0x2eece11, (0xc31f bytes) 2024-09-16T14:09:04.671 INFO:teuthology.orchestra.run.smithi079.stdout:5175 writesame 0xc857301 thru 0xc858a14 data_size 0xe(0x1714 bytes) 2024-09-16T14:09:04.671 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.671 INFO:teuthology.orchestra.run.smithi079.stdout:5177 read 0x67dc6eb thru 0x67e10f5 (0x4a0b bytes) 2024-09-16T14:09:04.684 INFO:teuthology.orchestra.run.smithi079.stdout:5178 write 0xee40089 thru 0xee4ac73 (0xabeb bytes) 2024-09-16T14:09:04.691 INFO:teuthology.orchestra.run.smithi079.stdout:5179 read 0x8445f8d thru 0x845334f (0xd3c3 bytes) 2024-09-16T14:09:04.691 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:04.688+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:04.696 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:04.696 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:04.696 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:04.696 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:04.705 INFO:teuthology.orchestra.run.smithi079.stdout:5181 trunc from 0xee4ac74 to 0x605c137 2024-09-16T14:09:04.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:04.739 INFO:teuthology.orchestra.run.smithi079.stdout:5183 trunc from 0x605c137 to 0x1df1669 2024-09-16T14:09:04.765 INFO:teuthology.orchestra.run.smithi079.stdout:5184 read 0xad5232 thru 0xadaad5 (0x58a4 bytes) 2024-09-16T14:09:04.765 INFO:teuthology.orchestra.run.smithi079.stdout:5186 read 0x855325 thru 0x85d870 (0x854c bytes) 2024-09-16T14:09:04.766 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.778 INFO:teuthology.orchestra.run.smithi079.stdout:5190 write 0x3863bbc thru 0x38703ce (0xc813 bytes) 2024-09-16T14:09:04.785 INFO:teuthology.orchestra.run.smithi079.stdout:5191 punch from 0xb14c85 to 0xb245ff, (0xf97a bytes) 2024-09-16T14:09:04.785 INFO:teuthology.orchestra.run.smithi079.stdout:5192 write 0x1a37631 thru 0x1a46bbc (0xf58c bytes) 2024-09-16T14:09:04.792 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:04.788+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:04.818 INFO:teuthology.orchestra.run.smithi079.stdout:5193 writesame 0xd1666de thru 0xd172aca data_size 0x3(0xc3ed bytes) 2024-09-16T14:09:04.818 INFO:teuthology.orchestra.run.smithi079.stdout:5194 read 0x2ca9af7 thru 0x2cb13b1 (0x78bb bytes) 2024-09-16T14:09:04.827 INFO:teuthology.orchestra.run.smithi079.stdout:5195 trunc from 0xd172acb to 0x6a866c5 2024-09-16T14:09:04.839 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.839 INFO:teuthology.orchestra.run.smithi079.stdout:5198 read 0x5b10a0e thru 0x5b10e9a (0x48d bytes) 2024-09-16T14:09:04.839 INFO:teuthology.orchestra.run.smithi079.stdout:5199 read 0x46eb1cf thru 0x46ef790 (0x45c2 bytes) 2024-09-16T14:09:04.842 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:04.843 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:04.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:04.861 INFO:teuthology.orchestra.run.smithi079.stdout:5202 write 0xdb095f2 thru 0xdb14e93 (0xb8a2 bytes) 2024-09-16T14:09:04.866 INFO:teuthology.orchestra.run.smithi079.stdout:5204 read 0xcf87ce8 thru 0xcf91af9 (0x9e12 bytes) 2024-09-16T14:09:04.867 INFO:teuthology.orchestra.run.smithi079.stdout:5205 writesame 0xcb91a5 thru 0xcc5ab3 data_size 0x3f(0xc90f bytes) 2024-09-16T14:09:04.871 INFO:teuthology.orchestra.run.smithi079.stdout:5208 writesame 0xb4e3217 thru 0xb4e7019 data_size 0x19(0x3e03 bytes) 2024-09-16T14:09:04.874 INFO:teuthology.orchestra.run.smithi079.stdout:5209 read 0x81c2296 thru 0x81c4a70 (0x27db bytes) 2024-09-16T14:09:04.874 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:04.874 INFO:teuthology.orchestra.run.smithi079.stdout:5212 read 0xb451160 thru 0xb45f955 (0xe7f6 bytes) 2024-09-16T14:09:04.880 INFO:teuthology.orchestra.run.smithi079.stdout:5214 trunc from 0xdb14e94 to 0xac0bbc3 2024-09-16T14:09:04.893 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:04.885+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:04.897 INFO:teuthology.orchestra.run.smithi079.stdout:5215 punch from 0x8571a1b to 0x85746cc, (0x2cb1 bytes) 2024-09-16T14:09:04.897 INFO:teuthology.orchestra.run.smithi079.stdout:5216 writesame 0x56f6086 thru 0x5701146 data_size 0x1(0xb0c1 bytes) 2024-09-16T14:09:04.951 INFO:teuthology.orchestra.run.smithi079.stdout:5217 writesame 0xc482094 thru 0xc4905cd data_size 0x3d(0xe53a bytes) 2024-09-16T14:09:04.951 INFO:teuthology.orchestra.run.smithi079.stdout:5220 read 0x36d8bb3 thru 0x36e18b2 (0x8d00 bytes) 2024-09-16T14:09:04.954 INFO:teuthology.orchestra.run.smithi079.stdout:5221 trunc from 0xc4905ce to 0xa34ec7b 2024-09-16T14:09:04.966 INFO:teuthology.orchestra.run.smithi079.stdout:5223 punch from 0x5a34ed8 to 0x5a3dee9, (0x9011 bytes) 2024-09-16T14:09:04.966 INFO:teuthology.orchestra.run.smithi079.stdout:5224 punch from 0x845725 to 0x85349c, (0xdd77 bytes) 2024-09-16T14:09:04.966 INFO:teuthology.orchestra.run.smithi079.stdout:5226 write 0x3c11dcf thru 0x3c18de2 (0x7014 bytes) 2024-09-16T14:09:04.972 INFO:teuthology.orchestra.run.smithi079.stdout:5227 read 0x547cff1 thru 0x5481a9c (0x4aac bytes) 2024-09-16T14:09:04.975 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.067453431999994, 2024-09-16T14:09:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:09:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:09:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:09:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:09:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:09:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:09:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:09:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:09:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:09:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:09:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:09:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:09:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:09:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:09:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:09:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:09:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:09:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:09:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:09:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.067430724000005, 2024-09-16T14:09:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:09:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:09:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:09:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.067406571999996, 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:09:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.067373630999995, 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:09:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.067348948000003, 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:09:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.067325718999996, 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:09:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.065479754999998, 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:09:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:09:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:09:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:09:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:09:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:09:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:09:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:09:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:09:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:09:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:09:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:09:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:09:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:09:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:09:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:09:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:09:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:09:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.065436891000004, 2024-09-16T14:09:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:09:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:09:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:09:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:09:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:09:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:09:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:09:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:09:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:09:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:09:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:09:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:09:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063977273000006, 2024-09-16T14:09:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:09:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063961504000005, 2024-09-16T14:09:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:09:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:09:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063945751999995, 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063930321000001, 2024-09-16T14:09:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:09:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:09:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:09:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:09:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:09:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:09:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:09:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:09:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:09:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:09:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:09:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:09:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:09:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:09:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:09:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:09:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:09:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:09:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:09:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:09:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063914682999993, 2024-09-16T14:09:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:09:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:09:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:09:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:09:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:09:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:09:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:09:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:09:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:09:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:09:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:09:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:09:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:09:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:09:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:09:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:09:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:09:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:09:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063899543999995, 2024-09-16T14:09:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:09:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:09:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:09:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:09:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063884707, 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:09:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063861703000001, 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:09:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063846478000002, 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:09:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063831293999996, 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:09:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063816048999996, 2024-09-16T14:09:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:09:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.063800891, 2024-09-16T14:09:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:09:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.119 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:05.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:5228 write 0xea99f86 thru 0xeaa8d99 (0xee14 bytes) 2024-09-16T14:09:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:5229 read 0x7880128 thru 0x788a01e (0x9ef7 bytes) 2024-09-16T14:09:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:5230 trunc from 0xeaa8d9a to 0x423cf63 2024-09-16T14:09:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:5231 writesame 0x54279fc thru 0x542f620 data_size 0x1(0x7c25 bytes) 2024-09-16T14:09:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:5234 read 0x371f9b9 thru 0x372b3b6 (0xb9fe bytes) 2024-09-16T14:09:05.122 INFO:teuthology.orchestra.run.smithi079.stdout:5235 write 0x7187730 thru 0x7196156 (0xea27 bytes) 2024-09-16T14:09:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:5236 writesame 0x511aa0a thru 0x511ec6f data_size 0x3(0x4266 bytes) 2024-09-16T14:09:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:5237 read 0x45d4051 thru 0x45d7aa4 (0x3a54 bytes) 2024-09-16T14:09:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:5238 write 0x1210697 thru 0x121d2fb (0xcc65 bytes) 2024-09-16T14:09:05.123 INFO:teuthology.orchestra.run.smithi079.stdout:5239 write 0x8b70ff6 thru 0x8b7ce22 (0xbe2d bytes) 2024-09-16T14:09:05.123 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:05.116+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:5240 writesame 0x9bab03b thru 0x9bac15d data_size 0x1(0x1123 bytes) 2024-09-16T14:09:05.131 INFO:teuthology.orchestra.run.smithi079.stdout:5242 read 0x427b15a thru 0x4286b54 (0xb9fb bytes) 2024-09-16T14:09:05.138 INFO:teuthology.orchestra.run.smithi079.stdout:5243 trunc from 0x9bac15e to 0x735380e 2024-09-16T14:09:05.162 INFO:teuthology.orchestra.run.smithi079.stdout:5244 write 0x5b9a598 thru 0x5ba9f8b (0xf9f4 bytes) 2024-09-16T14:09:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:05.166 INFO:teuthology.orchestra.run.smithi079.stdout:5246 read 0x2073258 thru 0x207fb35 (0xc8de bytes) 2024-09-16T14:09:05.167 INFO:teuthology.orchestra.run.smithi079.stdout:5247 read 0x41062f5 thru 0x4107194 (0xea0 bytes) 2024-09-16T14:09:05.168 INFO:teuthology.orchestra.run.smithi079.stdout:5248 read 0x68141da thru 0x681dda5 (0x9bcc bytes) 2024-09-16T14:09:05.192 INFO:teuthology.orchestra.run.smithi079.stdout:5249 write 0xda20438 thru 0xda26508 (0x60d1 bytes) 2024-09-16T14:09:05.198 INFO:teuthology.orchestra.run.smithi079.stdout:5250 writesame 0x83abd0b thru 0x83b7774 data_size 0x2(0xba6a bytes) 2024-09-16T14:09:05.219 INFO:teuthology.orchestra.run.smithi079.stdout:5251 writesame 0xccbc2af thru 0xccbc2d3 data_size 0x1(0x25 bytes) 2024-09-16T14:09:05.221 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:05.217+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:05.224 INFO:teuthology.orchestra.run.smithi079.stdout:5252 read 0xcb9de28 thru 0xcba9c6e (0xbe47 bytes) 2024-09-16T14:09:05.237 INFO:teuthology.orchestra.run.smithi079.stdout:5253 trunc from 0xda26509 to 0x6d7be21 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:184124 3.13 e117/112 3:c841e23c:::rbd_data.1136b1466cf.000000000000002e:head v 117'516, mlcod=117'514)", 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:05.240277+0000", 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.001691758, 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001736708, 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240277+0000", 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240277+0000", 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240277+0000", 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3200000000000004e-07 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240283+0000", 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0560000000000001e-06 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240298+0000", 2024-09-16T14:09:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4751e-05 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240304+0000", 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7440000000000004e-06 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240385+0000", 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1058999999999998e-05 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240431+0000", 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5921000000000003e-05 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.241952+0000", 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001521056 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:184125 3.13 e117/112 3:cc432700:::rbd_data.1136b1466cf.0000000000000033:head v 117'517, mlcod=117'514)", 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:05.240861+0000", 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.001106884, 2024-09-16T14:09:05.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001272947, 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240861+0000", 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240861+0000", 2024-09-16T14:09:05.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240863+0000", 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5090000000000001e-06 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240877+0000", 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.399e-05 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240917+0000", 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9848999999999997e-05 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.240937+0000", 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0653e-05 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.241075+0000", 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013801699999999999 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:05.241139+0000", 2024-09-16T14:09:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4065000000000004e-05 2024-09-16T14:09:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:05.251 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 2 2024-09-16T14:09:05.251 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:05.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:05.259 INFO:teuthology.orchestra.run.smithi079.stdout:5254 write 0x15157cb thru 0x151876d (0x2fa3 bytes) 2024-09-16T14:09:05.262 INFO:teuthology.orchestra.run.smithi079.stdout:5255 read 0xbd2d4f thru 0xbe002d (0xd2df bytes) 2024-09-16T14:09:05.264 INFO:teuthology.orchestra.run.smithi079.stdout:5256 write 0x1d541f thru 0x1dd646 (0x8228 bytes) 2024-09-16T14:09:05.288 INFO:teuthology.orchestra.run.smithi079.stdout:5257 write 0xc48e074 thru 0xc49970c (0xb699 bytes) 2024-09-16T14:09:05.297 INFO:teuthology.orchestra.run.smithi079.stdout:5258 punch from 0x6ff5873 to 0x6ff9c1c, (0x43a9 bytes) 2024-09-16T14:09:05.297 INFO:teuthology.orchestra.run.smithi079.stdout:5259 read 0x4356456 thru 0x436452a (0xe0d5 bytes) 2024-09-16T14:09:05.297 INFO:teuthology.orchestra.run.smithi079.stdout:5260 write 0xabac073 thru 0xabb19c4 (0x5952 bytes) 2024-09-16T14:09:05.306 INFO:teuthology.orchestra.run.smithi079.stdout:5263 read 0x1e06771 thru 0x1e12a94 (0xc324 bytes) 2024-09-16T14:09:05.307 INFO:teuthology.orchestra.run.smithi079.stdout:5264 write 0x331e4a4 thru 0x332cb33 (0xe690 bytes) 2024-09-16T14:09:05.316 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:05.316 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:05.316 INFO:teuthology.orchestra.run.smithi079.stdout:5267 read 0x7afda41 thru 0x7b005b0 (0x2b70 bytes) 2024-09-16T14:09:05.316 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:05.316 INFO:teuthology.orchestra.run.smithi079.stdout:5271 punch from 0x3b1ab59 to 0x3b2a106, (0xf5ad bytes) 2024-09-16T14:09:05.316 INFO:teuthology.orchestra.run.smithi079.stdout:5272 write 0x58bcca2 thru 0x58c97f1 (0xcb50 bytes) 2024-09-16T14:09:05.322 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:05.316+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:05.324 INFO:teuthology.orchestra.run.smithi079.stdout:5273 write 0x3e38ea4 thru 0x3e439fc (0xab59 bytes) 2024-09-16T14:09:05.327 INFO:teuthology.orchestra.run.smithi079.stdout:5274 read 0xb024cf4 thru 0xb03066a (0xb977 bytes) 2024-09-16T14:09:05.327 INFO:teuthology.orchestra.run.smithi079.stdout:5275 writesame 0x56ca21a thru 0x56d4701 data_size 0xc(0xa4e8 bytes) 2024-09-16T14:09:05.332 INFO:teuthology.orchestra.run.smithi079.stdout:5277 read 0x1bf44fd thru 0x1c03ae5 (0xf5e9 bytes) 2024-09-16T14:09:05.333 INFO:teuthology.orchestra.run.smithi079.stdout:5278 write 0x2d4119c thru 0x2d509ce (0xf833 bytes) 2024-09-16T14:09:05.356 INFO:teuthology.orchestra.run.smithi079.stdout:5279 trunc from 0xc49970d to 0x3540e 2024-09-16T14:09:05.365 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:05.365 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:05.365 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:05.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:05.405 INFO:teuthology.orchestra.run.smithi079.stdout:5280 punch from 0x15e5e to 0x1d5ba, (0x775c bytes) 2024-09-16T14:09:05.421 INFO:teuthology.orchestra.run.smithi079.stdout:5282 trunc from 0x3540e to 0x9c59a1e 2024-09-16T14:09:05.422 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:05.413+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:05.430 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:05.431 INFO:teuthology.orchestra.run.smithi079.stdout:5285 writesame 0x8e29f1e thru 0x8e38289 data_size 0xa(0xe36c bytes) 2024-09-16T14:09:05.441 INFO:teuthology.orchestra.run.smithi079.stdout:5287 write 0x8bf32d9 thru 0x8c01e67 (0xeb8f bytes) 2024-09-16T14:09:05.450 INFO:teuthology.orchestra.run.smithi079.stdout:5288 read 0x360c863 thru 0x361a4bc (0xdc5a bytes) 2024-09-16T14:09:05.450 INFO:teuthology.orchestra.run.smithi079.stdout:5290 read 0x3da42c thru 0x3e5d8c (0xb961 bytes) 2024-09-16T14:09:05.450 INFO:teuthology.orchestra.run.smithi079.stdout:5292 read 0x566b9ef thru 0x5675156 (0x9768 bytes) 2024-09-16T14:09:05.450 INFO:teuthology.orchestra.run.smithi079.stdout:5293 write 0x42485f3 thru 0x424a7a4 (0x21b2 bytes) 2024-09-16T14:09:05.458 INFO:teuthology.orchestra.run.smithi079.stdout:5294 read 0x58b5ed3 thru 0x58c2485 (0xc5b3 bytes) 2024-09-16T14:09:05.458 INFO:teuthology.orchestra.run.smithi079.stdout:5295 read 0x737e004 thru 0x738c697 (0xe694 bytes) 2024-09-16T14:09:05.458 INFO:teuthology.orchestra.run.smithi079.stdout:5296 punch from 0x1699d35 to 0x16a818d, (0xe458 bytes) 2024-09-16T14:09:05.458 INFO:teuthology.orchestra.run.smithi079.stdout:5297 read 0x890ceb5 thru 0x8918054 (0xb1a0 bytes) 2024-09-16T14:09:05.458 INFO:teuthology.orchestra.run.smithi079.stdout:5298 write 0x3f66777 thru 0x3f71b67 (0xb3f1 bytes) 2024-09-16T14:09:05.464 INFO:teuthology.orchestra.run.smithi079.stdout:5299 writesame 0x3df4037 thru 0x3dff4a0 data_size 0x7(0xb46a bytes) 2024-09-16T14:09:05.474 INFO:teuthology.orchestra.run.smithi079.stdout:5301 writesame 0x8dfe5c6 thru 0x8e096e2 data_size 0x1(0xb11d bytes) 2024-09-16T14:09:05.498 INFO:teuthology.orchestra.run.smithi079.stdout:5302 clone 44 order 20 su 262144 sc 12 2024-09-16T14:09:05.508 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:05.508 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:05.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:05.508 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:05.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.984879995999997, 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:09:05.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:09:05.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:09:05.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.979608657, 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:09:05.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:09:05.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:09:05.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.797100143, 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:09:05.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161246 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4096000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.793708006999999, 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062177214000000001, 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161246 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6199999999999999e-07 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710071+0000", 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.159e-06 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710074+0000", 2024-09-16T14:09:05.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:09:05.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710076+0000", 2024-09-16T14:09:05.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9439999999999999e-06 2024-09-16T14:09:05.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772126+0000", 2024-09-16T14:09:05.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062050007999999997 2024-09-16T14:09:05.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772183+0000", 2024-09-16T14:09:05.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6446999999999997e-05 2024-09-16T14:09:05.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772247+0000", 2024-09-16T14:09:05.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4399000000000005e-05 2024-09-16T14:09:05.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.534 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.534 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:05.534 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.793699611999999, 2024-09-16T14:09:05.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:09:05.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:09:05.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.789648333000001, 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.539 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:09:05.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.543 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.785895417000001, 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:09:05.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:09:05.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781815889000001, 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.549 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:09:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781799287, 2024-09-16T14:09:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:09:05.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:09:05.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781783311, 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:09:05.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.781765359, 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.777570228, 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:09:05.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:05.572 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.77747325, 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:05.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:09:05.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.577 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.777448709, 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:09:05.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:09:05.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.773891946999999, 2024-09-16T14:09:05.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:05.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:09:05.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:09:05.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:09:05.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:09:05.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:09:05.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.773874189000001, 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:09:05.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:09:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.773856525999999, 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:05.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:09:05.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.773839332, 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.598 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:09:05.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:05.602 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.770142914000001, 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:09:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 14.757812961999999, 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:05.607 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:05.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:09:05.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:05.612 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:05.612 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:05.613 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:05.614 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:05.608+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:05.714 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:05.705+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:05.731 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:05.732 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:05.732 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:05.732 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:05.733 INFO:teuthology.orchestra.run.smithi079.stdout:leaving image image_client.0-clone43 intact 2024-09-16T14:09:05.741 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:05.815 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:05.812+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:05.840 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:05.840 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:05.840 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:05.840 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:05.840 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:05.849 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:05.916 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:05.909+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:05.947 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.607401908, 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:09:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:09:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:09:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:09:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:09:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:09:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:09:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:09:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:09:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:09:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:09:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:09:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:09:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:09:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:09:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:09:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:09:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:09:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:09:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:09:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.607379511, 2024-09-16T14:09:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:09:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:09:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:09:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:09:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:09:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:09:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:09:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:09:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:09:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:09:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:09:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:09:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:09:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:09:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:09:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:09:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:09:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:09:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:09:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:09:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:09:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:09:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:09:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:09:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.6067728, 2024-09-16T14:09:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:09:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:09:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:09:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.60669847299999, 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:09:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:09:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:09:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.635773681000003, 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:09:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:09:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 95.007815100000002, 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:09:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:09:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:09:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.001 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 94.477654634000004, 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:06.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:06.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:09:06.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:09:06.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.830106103000006, 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.009 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:09:06.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:09:06.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.013 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.283761654999999, 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.014 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:09:06.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:09:06.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.283740979000001, 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:09:06.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:06.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:06.023 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.283698799, 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:06.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:09:06.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.283664573999999, 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:09:06.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:06.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.283634857999999, 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:09:06.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:09:06.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:06.037 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.283615196, 2024-09-16T14:09:06.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:09:06.038 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.038 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.038 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.038 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.038 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.038 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:09:06.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:09:06.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.274855070999998, 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.044 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:09:06.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.190128528999999, 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.049 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:06.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:09:06.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 25.545124992000002, 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:09:06.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:09:06.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:09:06.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.060 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 25.545038130999998, 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:06.061 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:09:06.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:09:06.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:06.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3557315120000002, 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.069 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:09:06.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:09:06.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.47050729000000002, 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:09:06.075 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:09:06.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:09:06.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:06.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:09:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:09:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:06.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:06.082 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:06.084 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:06.084 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:06.085 INFO:teuthology.orchestra.run.smithi079.stdout:5306 write 0xc9eeb58 thru 0xc9fc8c0 (0xdd69 bytes) 2024-09-16T14:09:06.085 INFO:teuthology.orchestra.run.smithi079.stdout:5307 punch from 0x8f2e7ce to 0x8f3a5c6, (0xbdf8 bytes) 2024-09-16T14:09:06.085 INFO:teuthology.orchestra.run.smithi079.stdout:5308 punch from 0xa4caae4 to 0xa4d3403, (0x891f bytes) 2024-09-16T14:09:06.085 INFO:teuthology.orchestra.run.smithi079.stdout:5309 punch from 0xad98d25 to 0xad9b1a8, (0x2483 bytes) 2024-09-16T14:09:06.085 INFO:teuthology.orchestra.run.smithi079.stdout:5310 trunc from 0xc9fc8c1 to 0xe1ff528 2024-09-16T14:09:06.085 INFO:teuthology.orchestra.run.smithi079.stdout:5311 trunc from 0xe1ff528 to 0x250128b 2024-09-16T14:09:06.085 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:06.080+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:06.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:06.100 INFO:teuthology.orchestra.run.smithi079.stdout:5314 trunc from 0x250128b to 0x81364ae 2024-09-16T14:09:06.110 INFO:teuthology.orchestra.run.smithi079.stdout:5316 write 0x6f7edf1 thru 0x6f823bb (0x35cb bytes) 2024-09-16T14:09:06.116 INFO:teuthology.orchestra.run.smithi079.stdout:5317 read 0xe340d3 thru 0xe428fe (0xe82c bytes) 2024-09-16T14:09:06.136 INFO:teuthology.orchestra.run.smithi079.stdout:5318 write 0xcf5ce81 thru 0xcf6869f (0xb81f bytes) 2024-09-16T14:09:06.142 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:06.143 INFO:teuthology.orchestra.run.smithi079.stdout:5320 write 0x6cfabb5 thru 0x6cfba6f (0xebb bytes) 2024-09-16T14:09:06.148 INFO:teuthology.orchestra.run.smithi079.stdout:5321 write 0x8663435 thru 0x8667617 (0x41e3 bytes) 2024-09-16T14:09:06.173 INFO:teuthology.orchestra.run.smithi079.stdout:5322 trunc from 0xcf686a0 to 0x3d2e42b 2024-09-16T14:09:06.184 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:06.177+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:06.189 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:06.189 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:06.190 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:06.192 INFO:teuthology.orchestra.run.smithi079.stdout:5323 read 0x2c76328 thru 0x2c76c09 (0x8e2 bytes) 2024-09-16T14:09:06.199 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:06.202 INFO:teuthology.orchestra.run.smithi079.stdout:5325 writesame 0x48289ff thru 0x4833483 data_size 0x1(0xaa85 bytes) 2024-09-16T14:09:06.202 INFO:teuthology.orchestra.run.smithi079.stdout:5326 read 0x2bf6c46 thru 0x2c00196 (0x9551 bytes) 2024-09-16T14:09:06.202 INFO:teuthology.orchestra.run.smithi079.stdout:5327 write 0x34ece4f thru 0x34f7402 (0xa5b4 bytes) 2024-09-16T14:09:06.220 INFO:teuthology.orchestra.run.smithi079.stdout:5329 writesame 0x6365af2 thru 0x636da43 data_size 0x2(0x7f52 bytes) 2024-09-16T14:09:06.220 INFO:teuthology.orchestra.run.smithi079.stdout:5330 punch from 0x3a7bc64 to 0x3a80bb6, (0x4f52 bytes) 2024-09-16T14:09:06.227 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:09:06.233 INFO:teuthology.orchestra.run.smithi079.stdout:5332 trunc from 0x636da44 to 0xe8fd1fc 2024-09-16T14:09:06.245 INFO:teuthology.orchestra.run.smithi079.stdout:5333 read 0xb31b004 thru 0xb31edb0 (0x3dad bytes) 2024-09-16T14:09:06.245 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:06.245 INFO:teuthology.orchestra.run.smithi079.stdout:5335 read 0x403385f thru 0x40382db (0x4a7d bytes) 2024-09-16T14:09:06.245 INFO:teuthology.orchestra.run.smithi079.stdout:5337 punch from 0xba33270 to 0xba384f4, (0x5284 bytes) 2024-09-16T14:09:06.245 INFO:teuthology.orchestra.run.smithi079.stdout:5338 read 0x4bcf8a6 thru 0x4bd0e4b (0x15a6 bytes) 2024-09-16T14:09:06.248 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:06.257 INFO:teuthology.orchestra.run.smithi079.stdout:5340 trunc from 0xe8fd1fc to 0x67add3c 2024-09-16T14:09:06.268 INFO:teuthology.orchestra.run.smithi079.stdout:5341 punch from 0x178e9e3 to 0x179695a, (0x7f77 bytes) 2024-09-16T14:09:06.268 INFO:teuthology.orchestra.run.smithi079.stdout:5342 write 0x5681835 thru 0x568c037 (0xa803 bytes) 2024-09-16T14:09:06.272 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:06.275 INFO:teuthology.orchestra.run.smithi079.stdout:5343 write 0x2ee20a1 thru 0x2ee9262 (0x71c2 bytes) 2024-09-16T14:09:06.281 INFO:teuthology.orchestra.run.smithi079.stdout:5344 writesame 0x3bcd110 thru 0x3bd35e1 data_size 0x3a(0x64d2 bytes) 2024-09-16T14:09:06.285 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:06.280+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:06.292 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:06.294 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:06.294 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:06.294 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:06.294 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:06.294 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:06.301 INFO:teuthology.orchestra.run.smithi079.stdout:5345 write 0x831aaa5 thru 0x8329b90 (0xf0ec bytes) 2024-09-16T14:09:06.304 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:06.306 INFO:teuthology.orchestra.run.smithi079.stdout:5348 punch from 0x58dd13d to 0x58e6718, (0x95db bytes) 2024-09-16T14:09:06.312 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:06.318 INFO:teuthology.orchestra.run.smithi079.stdout:5349 trunc from 0x8329b91 to 0xddca006 2024-09-16T14:09:06.333 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:06.337 INFO:teuthology.orchestra.run.smithi079.stdout:5350 trunc from 0xddca006 to 0x66c582d 2024-09-16T14:09:06.354 INFO:teuthology.orchestra.run.smithi079.stdout:5353 read 0x1a3a559 thru 0x1a465fa (0xc0a2 bytes) 2024-09-16T14:09:06.355 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:06.371 INFO:teuthology.orchestra.run.smithi079.stdout:5354 write 0x9392705 thru 0x939ed39 (0xc635 bytes) 2024-09-16T14:09:06.376 INFO:teuthology.orchestra.run.smithi079.stdout:5358 write 0x3cb965b thru 0x3cb9e80 (0x826 bytes) 2024-09-16T14:09:06.379 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:06.381 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:09:06.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:06.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:06.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:06.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:06.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:06.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:06.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:06.383 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6] out_osds: [7, 3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:09:06.383 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:09:06.383 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 4, 5, 6] 2024-09-16T14:09:06.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-16T14:09:06.384 INFO:teuthology.orchestra.run.smithi079.stdout:5359 read 0x6012df4 thru 0x6015a58 (0x2c65 bytes) 2024-09-16T14:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout:5362 read 0x114e3 thru 0x17151 (0x5c6f bytes) 2024-09-16T14:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout:5365 read 0x653a011 thru 0x6546205 (0xc1f5 bytes) 2024-09-16T14:09:06.385 INFO:teuthology.orchestra.run.smithi079.stdout:5366 read 0x2e35698 thru 0x2e44292 (0xebfb bytes) 2024-09-16T14:09:06.386 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:06.377+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:06.401 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:06.401 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:06.401 INFO:teuthology.orchestra.run.smithi079.stdout:5367 write 0xd0f5a53 thru 0xd0fbccf (0x627d bytes) 2024-09-16T14:09:06.406 INFO:teuthology.orchestra.run.smithi079.stdout:5368 write 0xa3f33fd thru 0xa3f5a9e (0x26a2 bytes) 2024-09-16T14:09:06.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:06.413 INFO:teuthology.orchestra.run.smithi079.stdout:5369 read 0x63a75fa thru 0x63b1e26 (0xa82d bytes) 2024-09-16T14:09:06.413 INFO:teuthology.orchestra.run.smithi079.stdout:5371 read 0x33138e5 thru 0x33204a4 (0xcbc0 bytes) 2024-09-16T14:09:06.413 INFO:teuthology.orchestra.run.smithi079.stdout:5372 read 0xb2687a9 thru 0xb277092 (0xe8ea bytes) 2024-09-16T14:09:06.413 INFO:teuthology.orchestra.run.smithi079.stdout:5373 writesame 0xa1e8567 thru 0xa1ec4d6 data_size 0xe(0x3f70 bytes) 2024-09-16T14:09:06.419 INFO:teuthology.orchestra.run.smithi079.stdout:5375 read 0x731a6c4 thru 0x731baad (0x13ea bytes) 2024-09-16T14:09:06.419 INFO:teuthology.orchestra.run.smithi079.stdout:5376 write 0x4a8d454 thru 0x4a8efba (0x1b67 bytes) 2024-09-16T14:09:06.425 INFO:teuthology.orchestra.run.smithi079.stdout:5378 writesame 0xa46eef8 thru 0xa47d506 data_size 0x1(0xe60f bytes) 2024-09-16T14:09:06.484 INFO:teuthology.orchestra.run.smithi079.stdout:5381 trunc from 0xd0fbcd0 to 0x3dbb828 2024-09-16T14:09:06.487 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:06.484+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:06.500 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:06.501 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:06.511 INFO:teuthology.orchestra.run.smithi079.stdout:5389 writesame 0x44f6f16 thru 0x44fdf90 data_size 0xd(0x707b bytes) 2024-09-16T14:09:06.517 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:06.517 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:06.521 INFO:teuthology.orchestra.run.smithi079.stdout:5390 trunc from 0x44fdf91 to 0xb0a076d 2024-09-16T14:09:06.526 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:06.545 INFO:teuthology.orchestra.run.smithi079.stdout:5392 write 0xd58059b thru 0xd580c41 (0x6a7 bytes) 2024-09-16T14:09:06.571 INFO:teuthology.orchestra.run.smithi079.stdout:5394 trunc from 0xd580c42 to 0x5c3138 2024-09-16T14:09:06.588 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:06.581+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:06.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:09:06.610 INFO:teuthology.orchestra.run.smithi079.stdout:5395 read 0x1b5846 thru 0x1c2d86 (0xd541 bytes) 2024-09-16T14:09:06.622 INFO:teuthology.orchestra.run.smithi079.stdout:5396 trunc from 0x5c3138 to 0x8274129 2024-09-16T14:09:06.632 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:06.632 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:06.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:06.643 INFO:teuthology.orchestra.run.smithi079.stdout:5397 write 0x9847c9c thru 0x9851c1f (0x9f84 bytes) 2024-09-16T14:09:06.646 INFO:teuthology.orchestra.run.smithi079.stdout:5398 clone 45 order 23 su 4194304 sc 11 2024-09-16T14:09:06.690 INFO:tasks.ceph.mon.a.smithi079.stderr:2024-09-16T14:09:06.688+0000 7f5ada529700 -1 mon.a@0(leader).osd e118 definitely_dead 0 2024-09-16T14:09:06.691 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:06.688+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:06.742 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:06.743 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:06.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:06.791 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:06.788+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:06.855 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:06.855 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:06.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:06.892 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:06.885+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:06.967 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:06.967 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:06.976 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:06.992 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:06.988+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:07.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:07.126 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:07.124+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:07.191 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:07.191 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:07.193 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:07.188+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:07.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:07.294 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:07.289+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:07.299 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:07.299 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:07.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:07.395 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:07.389+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:07.407 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:07.407 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:07.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:09:07.496 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:07.492+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:07.518 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:07.518 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:07.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:09:07.597 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:07.592+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:07.619 INFO:teuthology.orchestra.run.smithi079.stderr:marked out osd.1. 2024-09-16T14:09:07.623 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:09:07.638 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:07.639 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:07.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:09:07.698 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:07.696+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:07.799 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:07.793+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:07.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:07.902 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:07.902 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:07.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:07.924 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:07.893+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:08.001 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:07.993+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:08.006 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:08.007 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:08.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:08.102 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:08.100+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:08.117 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:08.117 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:08.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:08.203 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:08.200+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:08.227 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:08.227 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:08.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:08.304 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:08.300+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:08.337 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:08.338 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:08.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:08.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:08.405 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:08.400+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:08.468 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:08.468 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:08.477 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:08.527 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:08.521+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:08.532 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:09:08.549 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:08.569 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:08.586 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:08.586 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:08.587 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:08.596 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:08.606 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:08.612 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:08.604+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:08.635 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:08.654 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:08.674 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:08.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:09:08.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:08.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:08.676 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:08.677 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:08.677 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:08.677 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:08.677 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:08.697 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:08.697 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:08.697 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:08.697 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:08.697 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:08.706 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:08.707 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:08.701+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:08.806 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:08.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.465054259, 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:09:08.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:09:08.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:09:08.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.465031862, 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.813 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:09:08.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:09:08.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.818 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.464425151, 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.819 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:09:08.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:09:08.824 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.46435082399999, 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.825 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:09:08.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:09:08.830 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 101.493426032, 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:09:08.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.865467451000001, 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:09:08.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:09:08.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.335306985000003, 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:09:08.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:09:08.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:09:08.848 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:08.849 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.687758454000004, 2024-09-16T14:09:08.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:09:08.849 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:09:08.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:09:08.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101794 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.141414006000005, 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019827318, 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101794 2024-09-16T14:09:08.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660539+0000", 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660540+0000", 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660542+0000", 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7180000000000001e-06 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660549+0000", 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660555+0000", 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1709999999999999e-06 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680181+0000", 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019626009 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680270+0000", 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8669000000000001e-05 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680367+0000", 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6429999999999997e-05 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.14139333, 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:09:08.860 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.861 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:09:08.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.141351150000006, 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:09:08.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:09:08.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.141316924999998, 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.141287208999998, 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.878 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:09:08.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:08.882 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.141267546999998, 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:09:08.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.132507422000003, 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.047780879999998, 2024-09-16T14:09:08.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:08.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:09:08.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.402777343, 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:09:08.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:09:08.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.402690482000001, 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:09:08.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:09:08.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:09:08.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.2133838630000007, 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:09:08.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:09:08.916 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.3281596410000001, 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:08.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:08.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:08.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:08.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:08.922 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:08.924 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:08.924 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:08.920+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:09.025 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:09.020+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:09.027 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:09.027 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:09.027 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:09.036 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:09.126 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:09.124+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:09.137 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:09.137 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:09.138 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:09.138 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:09.138 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:09.146 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:09.227 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:09.221+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:09.250 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:09.250 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:09.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.415974869999999, 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:09.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:09.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.415958531000001, 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:09.257 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:09.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:09.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.415933365999997, 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:09.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:09.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.415029398999998, 2024-09-16T14:09:09.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:09.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.415020060000003, 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.274 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:09.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.414729938000001, 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:09.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:09.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.414511244000003, 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:09.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:09.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.408117664999999, 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:09.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:09.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.404206102000003, 2024-09-16T14:09:09.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:09.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:09.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:09.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.404174896999997, 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:09.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:09.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.403355457000004, 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:09.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:09.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.403310648000001, 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.312 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:09.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:09.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:09.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:09.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:09.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.400301394000003, 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:09.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:09.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:09.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.400284434, 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:09.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.329 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.400272938000001, 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:09.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:09.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.395526027999999, 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:09.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:09.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.387893396999999, 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:09.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:09.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:09.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.387881190000002, 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.346 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:09.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.385626275, 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:09.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:09.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:09.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:09.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.385402024000001, 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:09.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:09.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.362 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:09.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:09.364 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:09.357+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:09.465 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:09.457+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:09.513 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:09.513 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:09.514 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:09.514 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:09.522 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:09.566 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:09.561+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:09.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:09:09.637 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:09.637 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:09.637 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:09.637 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:09.637 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:09.653 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:09.678 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:09.676+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:09.759 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone44 from 0x9851c20 (overlap 0x5c3138) to 0x0 2024-09-16T14:09:09.768 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:09.764+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.396833491000002, 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.774 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:09:09.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:09:09.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.392902073000002, 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.783 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.392877880999997, 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:09:09.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.389158610000003, 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:09:09.788 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:09:09.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.385241845000003, 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:09.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:09:09.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.380859565999998, 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.802 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.380835724999997, 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:09:09.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.380820536999998, 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:09.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:09:09.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.380805950999999, 2024-09-16T14:09:09.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:09.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:09:09.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.380702300999999, 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.817 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:09:09.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.380674575, 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:09:09.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.378000958999998, 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.827 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.377974244999997, 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.832 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.836 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.377934078000003, 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:09:09.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.377844363000001, 2024-09-16T14:09:09.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:09:09.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:09:09.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.377435558999998, 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:09:09.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.377335375000001, 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:09:09.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.855 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.369238005, 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:09.861 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.367069712999999, 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:09.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:09:09.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:09:09.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.217508252, 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:09.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:09:09.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:09:09.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:09.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:09:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:09:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:09.874 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:09.874 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:09.875 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:09.876 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:09.869+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:09.917 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-16T14:09:09.976 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:09.972+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:10.039 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:10.039 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:10.039 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:10.039 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:10.052 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:10.077 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:10.072+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:10.157 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:10.158 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:10.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:10.178 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:10.176+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:10.226 INFO:teuthology.orchestra.run.smithi079.stdout:5399 punch from 0x1799d73 to 0x179ffad, (0x623a bytes) 2024-09-16T14:09:10.247 INFO:teuthology.orchestra.run.smithi079.stdout:5400 write 0xbd39bdc thru 0xbd3b4b7 (0x18dc bytes) 2024-09-16T14:09:10.255 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:10.255 INFO:teuthology.orchestra.run.smithi079.stdout:5402 punch from 0x983176e to 0x9836110, (0x49a2 bytes) 2024-09-16T14:09:10.255 INFO:teuthology.orchestra.run.smithi079.stdout:5403 read 0x77c4409 thru 0x77c5c4e (0x1846 bytes) 2024-09-16T14:09:10.255 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:10.255 INFO:teuthology.orchestra.run.smithi079.stdout:5405 punch from 0x62111fb to 0x6214803, (0x3608 bytes) 2024-09-16T14:09:10.255 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:10.269 INFO:teuthology.orchestra.run.smithi079.stdout:5408 writesame 0xd0d03e6 thru 0xd0d2ed3 data_size 0x23(0x2aee bytes) 2024-09-16T14:09:10.270 INFO:teuthology.orchestra.run.smithi079.stdout:5409 read 0x60eafaa thru 0x60ee643 (0x369a bytes) 2024-09-16T14:09:10.270 INFO:teuthology.orchestra.run.smithi079.stdout:5410 write 0x896cf89 thru 0x896ff19 (0x2f91 bytes) 2024-09-16T14:09:10.278 INFO:teuthology.orchestra.run.smithi079.stdout:5413 punch from 0x82ca8b0 to 0x82d86c3, (0xde13 bytes) 2024-09-16T14:09:10.278 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:10.276+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:10.281 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.867950854, 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 81.029988531000001, 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.502749543, 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.502726312, 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.502703365000002, 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.502686855999997, 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.502669230000002, 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.496753486000003, 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.496736081000002, 2024-09-16T14:09:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.496381714999998, 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.493237856, 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.49322257, 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.493206161000003, 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.493130964000002, 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.493033416999999, 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.493008176000004, 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.492979122000001, 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.489715132999997, 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.489696932000001, 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:10.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.489657754, 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:10.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:10.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:10.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.379 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:10.380 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:10.380 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:10.381 INFO:teuthology.orchestra.run.smithi079.stdout:5415 trunc from 0xd0d2ed4 to 0xbc9af14 2024-09-16T14:09:10.381 INFO:teuthology.orchestra.run.smithi079.stdout:5417 write 0x3c80d00 thru 0x3c843e9 (0x36ea bytes) 2024-09-16T14:09:10.381 INFO:teuthology.orchestra.run.smithi079.stdout:5418 write 0xb49950d thru 0xb49e397 (0x4e8b bytes) 2024-09-16T14:09:10.382 INFO:teuthology.orchestra.run.smithi079.stdout:5419 trunc from 0xbc9af14 to 0x7fe2de7 2024-09-16T14:09:10.382 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:09:10.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:10.383 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:10.373+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:10.385 INFO:teuthology.orchestra.run.smithi079.stdout:5420 trunc from 0x7fe2de7 to 0xd401fbd 2024-09-16T14:09:10.413 INFO:teuthology.orchestra.run.smithi079.stdout:5421 trunc from 0xd401fbd to 0x462552e 2024-09-16T14:09:10.481 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:10.481 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:10.482 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:10.473+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:10.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:10.583 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:10.580+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:10.645 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:10.645 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:10.645 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:10.645 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:10.645 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:10.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:10.683 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:10.677+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:10.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.543883178999998, 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:10.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183730136000001, 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:10.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.794 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183721136999999, 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183712786000001, 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:10.799 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:10.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:10.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183680102, 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:10.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:10.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183670865, 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:10.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.813 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183662333000001, 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:10.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:10.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183653755999998, 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:10.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183644395000002, 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.823 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183636282999998, 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:10.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.18362458, 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:10.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:10.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:10.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183615967000001, 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:10.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:10.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183607860999999, 2024-09-16T14:09:10.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:10.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:10.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183599519000001, 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:10.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:10.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183590980999998, 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:10.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:10.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183582649000002, 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:10.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183574406999998, 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:10.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:10.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183564843999999, 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:10.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:10.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:10.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183555463000001, 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:10.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:10.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.183545821999999, 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:10.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:10.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:10.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:10.877 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:10.878 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:10.777+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:10.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:09:10.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:09:10.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:10.885 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:10.880+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:10.899 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:10.936 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:10.967 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:10.986 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:10.977+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:10.992 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:10.996 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:10.996 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:09:11.020 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:11.049 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:11.051 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:09:11.051 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:11.051 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:11.051 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:11.051 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:11.051 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:11.052 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:11.052 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:11.086 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:11.084+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:11.109 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:11.110 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:09:11.187 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:11.184+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:11.211 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:11.212 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:11.288 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:11.281+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:11.385 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:11.385 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.388 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:11.381+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:11.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:11.421 INFO:teuthology.orchestra.run.smithi079.stdout:5422 write 0x4db177a thru 0x4dbe14e (0xc9d5 bytes) 2024-09-16T14:09:11.424 INFO:teuthology.orchestra.run.smithi079.stdout:5423 read 0x45b3a15 thru 0x45bbf93 (0x857f bytes) 2024-09-16T14:09:11.425 INFO:teuthology.orchestra.run.smithi079.stdout:5424 read 0xe04b9d thru 0xe0a2b9 (0x571d bytes) 2024-09-16T14:09:11.445 INFO:teuthology.orchestra.run.smithi079.stdout:5425 write 0x9b38bce thru 0x9b443ed (0xb820 bytes) 2024-09-16T14:09:11.490 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:11.481+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:11.499 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:11.500 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:11.591 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:11.585+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:11.617 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:11.617 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:11.692 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:11.685+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:11.736 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:11.736 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:11.793 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:11.787+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:11.873 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:11.873 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:11.918 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:11.915+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:11.987 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:11.987 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:11.995 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:11.989+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:11.999 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:12.096 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:12.091+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:12.115 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:12.115 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:12.125 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:12.196 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:12.191+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:12.231 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:12.231 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:12.241 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:12.297 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:12.291+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:12.343 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:12.344 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:12.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:12.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:09:12.388 INFO:teuthology.orchestra.run.smithi079.stdout:5427 punch from 0x1373684 to 0x137d11f, (0x9a9b bytes) 2024-09-16T14:09:12.388 INFO:teuthology.orchestra.run.smithi079.stdout:5428 read 0x409793e thru 0x40a2ab6 (0xb179 bytes) 2024-09-16T14:09:12.388 INFO:teuthology.orchestra.run.smithi079.stdout:5429 read 0x685090c thru 0x6854fb3 (0x46a8 bytes) 2024-09-16T14:09:12.398 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:12.389+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:12.460 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:12.461 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:12.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:12.498 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:12.495+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:12.599 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:12.595+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:12.630 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:12.631 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6] out_osds: [7, 3, 1] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:09:12.631 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:09:12.631 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-16T14:09:12.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-16T14:09:12.700 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:12.693+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:12.731 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:12.732 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:12.740 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:12.801 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:12.793+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:12.886 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:12.887 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:12.895 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:12.902 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:12.899+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:12.935 INFO:tasks.ceph.mon.a.smithi079.stderr:2024-09-16T14:09:12.931+0000 7f5ada529700 -1 mon.a@0(leader).osd e124 definitely_dead 0 2024-09-16T14:09:12.998 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:12.998 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:13.003 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:12.997+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:13.007 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:13.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:13.104 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:13.097+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:13.151 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:13.151 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:13.160 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:13.206 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:13.199+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:13.206 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:09:13.226 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:13.246 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:13.256 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:13.256 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:13.265 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:13.267 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:13.288 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:13.307 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:13.303+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:13.309 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:13.330 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:13.351 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:13.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:09:13.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:13.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:13.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:13.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:13.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:13.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:13.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:13.360 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:13.361 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:13.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:marked in osd.1. 2024-09-16T14:09:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:09:13.397 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-16T14:09:13.438 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:13.435+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.04829961900001, 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:13.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:13.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:13.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:13.469 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.470 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:13.470 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:13.470 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.210337296000006, 2024-09-16T14:09:13.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:13.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:13.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.683098307999998, 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.477 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.481 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.683075076999998, 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:13.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:13.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.68305213, 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:13.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.683035621000002, 2024-09-16T14:09:13.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:13.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:13.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.494 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.683017995, 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.495 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:13.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:13.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.677102251000001, 2024-09-16T14:09:13.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:13.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.677084846, 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:13.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.676730480000003, 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:13.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.513 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.673586620999998, 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:13.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.673571334999998, 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.518 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:13.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:13.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.673554926000001, 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.523 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:13.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:13.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:13.527 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.673479729, 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:13.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:13.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:13.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.673382181999997, 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:13.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.673356941000002, 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:13.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.673327886999999, 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.541 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:13.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.670063898000002, 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.546 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:13.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.550 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.670045696999999, 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:13.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.670006518999998, 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:13.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:13.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:13.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:13.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:13.560 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:13.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:13.562 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:13.557+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:13.662 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:13.659+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:13.672 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:13.673 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:13.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:13.763 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:13.759+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:13.828 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:13.829 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:13.829 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:13.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:13.882 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:13.879+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:13.940 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.700343117999999, 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:13.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:13.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340190074999999, 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.946 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:13.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340181076, 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340172724999999, 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340140041000002, 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340130804000001, 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340122271999999, 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340113694999999, 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340104333999999, 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340096222, 2024-09-16T14:09:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340084519000001, 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340075905999999, 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.3400678, 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340059457999999, 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340050919999999, 2024-09-16T14:09:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340042587999999, 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340034345999999, 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340024783, 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340015401999999, 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.340005761, 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.037 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:14.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:09:14.039 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:14.037+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:14.140 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:14.137+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:14.142 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:14.142 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:14.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:09:14.240 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:14.233+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:14.247 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:14.247 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:14.247 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:14.248 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:14.248 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:14.257 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:09:14.341 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:14.337+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:14.357 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.357 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45201 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2644992~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443346977000004, 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0303560519999999, 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45201 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909733+0000", 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909734+0000", 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7099999999999998e-07 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909735+0000", 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.423e-06 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909739+0000", 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8120000000000001e-06 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909741+0000", 2024-09-16T14:09:14.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1330000000000002e-06 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912118+0000", 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023770229999999998 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912121+0000", 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0299999999999998e-06 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939865+0000", 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0277440499999999 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.939951+0000", 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5964999999999994e-05 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940089+0000", 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000138345 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.362 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45202 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2657280~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443337249999999, 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0306092499999999, 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.363 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45202 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909743+0000", 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7500000000000001e-07 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909744+0000", 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1170000000000001e-06 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909748+0000", 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.253e-06 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909750+0000", 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3020000000000002e-06 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912142+0000", 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002391687 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912144+0000", 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3819999999999998e-06 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940117+0000", 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.027972433 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940209+0000", 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2460000000000006e-05 2024-09-16T14:09:14.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940352+0000", 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014334100000000001 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45203 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2669568~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443313687, 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030769018, 2024-09-16T14:09:14.368 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45203 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909767+0000", 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.67e-07 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909768+0000", 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3319999999999999e-06 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909772+0000", 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4759999999999998e-06 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909774+0000", 2024-09-16T14:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5170000000000002e-06 2024-09-16T14:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912166+0000", 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023918989999999998 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912169+0000", 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0970000000000002e-06 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940391+0000", 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028221381 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940452+0000", 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1041000000000001e-05 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940536+0000", 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4007999999999999e-05 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45204 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2681856~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443286447999995, 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03110218, 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45204 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909794+0000", 2024-09-16T14:09:14.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5400000000000002e-07 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909795+0000", 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909799+0000", 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909801+0000", 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3190000000000002e-06 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912189+0000", 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023875860000000001 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912191+0000", 2024-09-16T14:09:14.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3640000000000002e-06 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940582+0000", 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.028390479 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940701+0000", 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011941 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940896+0000", 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019503000000000001 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45205 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2694144~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443277034999994, 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313702650000001, 2024-09-16T14:09:14.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45205 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909803+0000", 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909804+0000", 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6899999999999999e-07 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909805+0000", 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0690000000000001e-06 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909808+0000", 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3009999999999998e-06 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909810+0000", 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2199999999999999e-06 2024-09-16T14:09:14.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912213+0000", 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403053 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912216+0000", 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.940954+0000", 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0287377289999999 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941059+0000", 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000105263 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941174+0000", 2024-09-16T14:09:14.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114366 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45206 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2706432~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443252716000003, 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0315083, 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45206 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909828+0000", 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8799999999999998e-07 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909829+0000", 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.28e-06 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909833+0000", 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4989999999999999e-06 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909836+0000", 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8480000000000001e-06 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912239+0000", 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002403634 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912242+0000", 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3740000000000001e-06 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941211+0000", 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0289698030000001 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941258+0000", 2024-09-16T14:09:14.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6996000000000002e-05 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941336+0000", 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7577999999999995e-05 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45207 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2718720~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:09:14.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443233109999994, 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0317047720000001, 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45207 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:09:14.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909847+0000", 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909848+0000", 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3099999999999999e-07 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909849+0000", 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.916e-06 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909859+0000", 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4399999999999994e-06 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909863+0000", 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2919999999999997e-06 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912263+0000", 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023995259999999999 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912265+0000", 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941368+0000", 2024-09-16T14:09:14.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029102454 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941436+0000", 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8208999999999997e-05 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941552+0000", 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116251 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45208 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2731008~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443212922000001, 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0319403279999999, 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45208 2024-09-16T14:09:14.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909867+0000", 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909868+0000", 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8200000000000001e-07 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909876+0000", 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2069999999999995e-06 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909882+0000", 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4230000000000002e-06 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909886+0000", 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8800000000000001e-06 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912287+0000", 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024002659999999999 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912289+0000", 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3479999999999998e-06 2024-09-16T14:09:14.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941572+0000", 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0292826669999999 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941635+0000", 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3227999999999998e-05 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941808+0000", 2024-09-16T14:09:14.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017292699999999999 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45209 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2743296~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443189653000005, 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032152993, 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45209 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909891+0000", 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909893+0000", 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909899+0000", 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.049e-06 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909912+0000", 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2445e-05 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912309+0000", 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023978099999999998 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912312+0000", 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3920000000000001e-06 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941829+0000", 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0295171569999999 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.941922+0000", 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2748e-05 2024-09-16T14:09:14.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942044+0000", 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000122021 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45210 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2755584~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443159402999996, 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032386281, 2024-09-16T14:09:14.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45210 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909921+0000", 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4799999999999999e-07 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909923+0000", 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4670000000000001e-06 2024-09-16T14:09:14.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909927+0000", 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-06 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909933+0000", 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5599999999999999e-06 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912333+0000", 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023999450000000001 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912335+0000", 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3269999999999999e-06 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942075+0000", 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.029739655 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942166+0000", 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1329999999999995e-05 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942307+0000", 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014081900000000001 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.412 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45211 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2767872~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443142651000002, 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0325443919999999, 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45211 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909938+0000", 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6300000000000001e-07 2024-09-16T14:09:14.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909940+0000", 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9910000000000001e-06 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909947+0000", 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909951+0000", 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4109999999999998e-06 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912356+0000", 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024047080000000002 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912358+0000", 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3580000000000001e-06 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942351+0000", 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0299929249999999 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942408+0000", 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6793000000000003e-05 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942482+0000", 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4155999999999999e-05 2024-09-16T14:09:14.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45212 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2780160~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443125386999995, 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032651789, 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45212 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909955+0000", 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.84e-07 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909957+0000", 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909964+0000", 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0740000000000004e-06 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909968+0000", 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8959999999999996e-06 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912380+0000", 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00241201 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912383+0000", 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.367e-06 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942499+0000", 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0301165940000001 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942541+0000", 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1777999999999999e-05 2024-09-16T14:09:14.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942607+0000", 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5675999999999999e-05 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45213 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2792448~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443086550999993, 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032764308, 2024-09-16T14:09:14.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45213 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909994+0000", 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3599999999999999e-07 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.909996+0000", 2024-09-16T14:09:14.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2840000000000001e-06 2024-09-16T14:09:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910009+0000", 2024-09-16T14:09:14.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2661e-05 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910014+0000", 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1769999999999997e-06 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912403+0000", 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023884499999999999 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912405+0000", 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3539999999999998e-06 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942627+0000", 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0302222240000001 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942690+0000", 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2649000000000003e-05 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942758+0000", 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8072999999999997e-05 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45214 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2804736~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443051722999996, 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0330120629999999, 2024-09-16T14:09:14.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45214 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910029+0000", 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9099999999999999e-07 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910031+0000", 2024-09-16T14:09:14.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3870000000000002e-06 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910044+0000", 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2431e-05 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910053+0000", 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4450000000000002e-06 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912426+0000", 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002373259 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912429+0000", 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3470000000000001e-06 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942795+0000", 2024-09-16T14:09:14.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030365805 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.942898+0000", 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103008 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943041+0000", 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014299000000000001 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.436 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45215 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2817024~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.443015837000004, 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0331929099999999, 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45215 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910064+0000", 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910065+0000", 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1199999999999998e-07 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910067+0000", 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3070000000000001e-06 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910079+0000", 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1649e-05 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910083+0000", 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.839e-06 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912449+0000", 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023658519999999999 2024-09-16T14:09:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912451+0000", 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943081+0000", 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030630312 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943142+0000", 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0714000000000002e-05 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943257+0000", 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000115501 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45216 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2829312~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.442993244999997, 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03355211, 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45216 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910087+0000", 2024-09-16T14:09:14.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910090+0000", 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0530000000000001e-06 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910096+0000", 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2779999999999999e-06 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910099+0000", 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912473+0000", 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023734770000000001 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912476+0000", 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1030000000000002e-06 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943347+0000", 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.030871525 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943449+0000", 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101285 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943639+0000", 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000190452 2024-09-16T14:09:14.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45217 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2841600~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.442964173000007, 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.033752773, 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.448 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45217 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910116+0000", 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910117+0000", 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3099999999999998e-07 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910119+0000", 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1830000000000001e-06 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910125+0000", 2024-09-16T14:09:14.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3350000000000003e-06 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910130+0000", 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4299999999999999e-06 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912497+0000", 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002367844 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912500+0000", 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3489999999999999e-06 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943684+0000", 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0311840299999999 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943756+0000", 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2397999999999994e-05 2024-09-16T14:09:14.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943869+0000", 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112773 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45218 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2853888~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.442945827000003, 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.03400563, 2024-09-16T14:09:14.453 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45218 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910134+0000", 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910135+0000", 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7599999999999998e-07 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910137+0000", 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910144+0000", 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910148+0000", 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1060000000000002e-06 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912522+0000", 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023739809999999998 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912525+0000", 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943899+0000", 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0313748170000001 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.943995+0000", 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5055000000000004e-05 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944140+0000", 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000145594 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.458 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45219 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2866176~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.442928488000007, 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0341833810000001, 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45219 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910152+0000", 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910154+0000", 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910160+0000", 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7590000000000003e-06 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910163+0000", 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4479999999999999e-06 2024-09-16T14:09:14.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912547+0000", 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023839070000000002 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912550+0000", 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944164+0000", 2024-09-16T14:09:14.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031614415 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944231+0000", 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6982000000000003e-05 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944335+0000", 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000104212 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:45220 3.13 3:cfe677fa:::rbd_data.113681563d6c.000000000000000d:15 [read 2878464~1024] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.442912565, 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.034436141, 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 45220 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.464 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910168+0000", 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8299999999999998e-07 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910176+0000", 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4329999999999998e-06 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910183+0000", 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5530000000000004e-06 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.910187+0000", 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9319999999999998e-06 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912562+0000", 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0023753929999999999 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.912565+0000", 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3549999999999999e-06 2024-09-16T14:09:14.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944362+0000", 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.031797393 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944459+0000", 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7038000000000002e-05 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.944604+0000", 2024-09-16T14:09:14.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144661 2024-09-16T14:09:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.469 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:14.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:14.471 INFO:teuthology.orchestra.run.smithi079.stdout:5430 writesame 0xe2a29c0 thru 0xe2a5dfc data_size 0x1(0x343d bytes) 2024-09-16T14:09:14.471 INFO:teuthology.orchestra.run.smithi079.stdout:5431 read 0xb649d2e thru 0xb657551 (0xd824 bytes) 2024-09-16T14:09:14.472 INFO:teuthology.orchestra.run.smithi079.stdout:5432 punch from 0x9c8fe5f to 0x9c9e281, (0xe422 bytes) 2024-09-16T14:09:14.472 INFO:teuthology.orchestra.run.smithi079.stdout:5433 write 0x4b1fa6a thru 0x4b2b720 (0xbcb7 bytes) 2024-09-16T14:09:14.472 INFO:teuthology.orchestra.run.smithi079.stdout:5434 punch from 0x8ce283c to 0x8ced03d, (0xa801 bytes) 2024-09-16T14:09:14.472 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:14.472 INFO:teuthology.orchestra.run.smithi079.stdout:5438 trunc from 0xe2a5dfd to 0x4d6655b 2024-09-16T14:09:14.472 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:14.467+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:14.571 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:14.569+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:14.622 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:14.622 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:14.623 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:14.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:14.672 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:14.669+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:14.733 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:14.733 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:14.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:14.772 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:14.769+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:14.842 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.933961517, 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:09:14.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:09:14.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.933938808999997, 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:14.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:09:14.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:09:14.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.933914657000003, 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:09:14.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.933881716000002, 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:09:14.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:14.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:14.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:09:14.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:09:14.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:09:14.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:09:14.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.933857032999995, 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:09:14.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:09:14.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.933833804000002, 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:09:14.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:09:14.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.931987840000005, 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:09:14.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.931944975999997, 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:14.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:09:14.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:09:14.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930485357999999, 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:09:14.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:09:14.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930469588999998, 2024-09-16T14:09:14.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:14.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:09:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930453837000002, 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:09:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930438405999993, 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:09:14.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:09:14.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930422768, 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:09:14.918 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:14.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:09:14.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.924 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930407629000001, 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:09:14.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:09:14.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930392792000006, 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.931 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:14.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:09:14.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:09:14.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:09:14.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930369787999993, 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.938 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:09:14.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.938 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:09:14.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930354562999995, 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.944 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:14.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:09:14.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930339379000003, 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:09:14.950 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:14.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:09:14.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:09:14.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.956 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930324134000003, 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.957 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:09:14.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.962 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.930308976000006, 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:14.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:09:14.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:09:14.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:14.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:09:14.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:09:14.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.969 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:14.969 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:14.969 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:14.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:14.971 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:14.969+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:15.072 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:15.067+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:15.078 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:15.078 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:15.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3.3cd9f411 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.65815861399999998, 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.65820475700000003, 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:09:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:09:15.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:15.083 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-16T14:09:15.084 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:15.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:15.173 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:15.169+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:15.188 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:15.189 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:15.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:15.274 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:15.271+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:15.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.777354086000003, 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:09:15.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:09:15.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.305 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.772082746999999, 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.306 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:09:15.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:09:15.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:15.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.312 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.589574233, 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:09:15.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161246 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4096000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.586182096999998, 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062177214000000001, 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161246 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710070+0000", 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6199999999999999e-07 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710071+0000", 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.159e-06 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710074+0000", 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9950000000000001e-06 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710076+0000", 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9439999999999999e-06 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772126+0000", 2024-09-16T14:09:15.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062050007999999997 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772183+0000", 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6446999999999997e-05 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772247+0000", 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4399000000000005e-05 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.321 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.586173702, 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.322 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:09:15.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:15.326 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.582122423000001, 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:15.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:09:15.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:09:15.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.578369507000001, 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.331 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:15.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.574289979, 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:09:15.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:09:15.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.574273377000001, 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:09:15.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:09:15.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.574257401000001, 2024-09-16T14:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.349 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.574239449, 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.570044318000001, 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:15.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:09:15.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.569947339999999, 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.569922799, 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:09:15.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.566366037000002, 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:15.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.372 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.566348279, 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:09:15.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.566330615999998, 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:09:15.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:09:15.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:15.381 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.566313422, 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:09:15.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.562617004, 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:09:15.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:09:15.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:15.390 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 24.550287052000002, 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:15.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:09:15.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:09:15.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:09:15.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:15.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:15.395 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:15.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:15.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:15.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:09:15.398 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:15.389+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:15.421 INFO:teuthology.orchestra.run.smithi079.stdout:5439 punch from 0x1aca6d8 to 0x1ad084b, (0x6173 bytes) 2024-09-16T14:09:15.442 INFO:teuthology.orchestra.run.smithi079.stdout:5440 write 0xc5c69d3 thru 0xc5cbfb8 (0x55e6 bytes) 2024-09-16T14:09:15.449 INFO:teuthology.orchestra.run.smithi079.stdout:5441 write 0x779c168 thru 0x77a203f (0x5ed8 bytes) 2024-09-16T14:09:15.467 INFO:teuthology.orchestra.run.smithi079.stdout:5443 write 0xcfa6e1a thru 0xcfa84e5 (0x16cc bytes) 2024-09-16T14:09:15.475 INFO:teuthology.orchestra.run.smithi079.stdout:5446 read 0x6e58cc3 thru 0x6e5c2fd (0x363b bytes) 2024-09-16T14:09:15.475 INFO:teuthology.orchestra.run.smithi079.stdout:5447 writesame 0x871460a thru 0x8716dc2 data_size 0x1(0x27b9 bytes) 2024-09-16T14:09:15.485 INFO:teuthology.orchestra.run.smithi079.stdout:5448 write 0x1aca045 thru 0x1ad37c2 (0x977e bytes) 2024-09-16T14:09:15.487 INFO:teuthology.orchestra.run.smithi079.stdout:5449 writesame 0x7d477ba thru 0x7d54e5e data_size 0x1(0xd6a5 bytes) 2024-09-16T14:09:15.497 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:15.495+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:15.507 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:09:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.015303908, 2024-09-16T14:09:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015354959, 2024-09-16T14:09:15.507 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:09:15.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:09:15.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:09:15.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:09:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:09:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:09:15.512 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T14:09:15.512 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:15.517 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:15.529 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:09:15.542 INFO:teuthology.orchestra.run.smithi079.stdout:5450 writesame 0xd5a8190 thru 0xd5b2d04 data_size 0x3(0xab75 bytes) 2024-09-16T14:09:15.542 INFO:teuthology.orchestra.run.smithi079.stdout:5451 write 0xbb2974b thru 0xbb2b65a (0x1f10 bytes) 2024-09-16T14:09:15.550 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:15.562 INFO:teuthology.orchestra.run.smithi079.stdout:5452 trunc from 0xd5b2d05 to 0x4e62f34 2024-09-16T14:09:15.574 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:15.594 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:15.595 INFO:teuthology.orchestra.run.smithi079.stdout:5453 trunc from 0x4e62f34 to 0xa33512e 2024-09-16T14:09:15.598 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:15.593+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:15.606 INFO:teuthology.orchestra.run.smithi079.stdout:5455 write 0xa3591f thru 0xa415cb (0xbcad bytes) 2024-09-16T14:09:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:15.612 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:15.612 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:15.615 INFO:teuthology.orchestra.run.smithi079.stdout:5456 read 0x47edaa5 thru 0x47f8c00 (0xb15c bytes) 2024-09-16T14:09:15.617 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:15.622 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:15.626 INFO:teuthology.orchestra.run.smithi079.stdout:5457 trunc from 0xa33512e to 0x380b379 2024-09-16T14:09:15.644 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:15.648 INFO:teuthology.orchestra.run.smithi079.stdout:5458 writesame 0x3ba3010 thru 0x3badab3 data_size 0x43(0xaaa4 bytes) 2024-09-16T14:09:15.650 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:15.657 INFO:teuthology.orchestra.run.smithi079.stdout:5462 trunc from 0x3badab4 to 0x96ca23 2024-09-16T14:09:15.670 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:15.693 INFO:teuthology.orchestra.run.smithi079.stdout:5464 read 0x251fdf thru 0x2521f1 (0x213 bytes) 2024-09-16T14:09:15.693 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:15.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:09:15.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:15.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:15.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:15.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:15.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:15.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:15.696 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:15.699 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:15.693+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:15.703 INFO:teuthology.orchestra.run.smithi079.stdout:5465 writesame 0xbf9492 thru 0xc040bf data_size 0x1(0xac2e bytes) 2024-09-16T14:09:15.706 INFO:teuthology.orchestra.run.smithi079.stdout:5467 punch from 0xa44892 to 0xa4eaca, (0xa238 bytes) 2024-09-16T14:09:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:15.722 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:15.732 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:15.732 INFO:teuthology.orchestra.run.smithi079.stdout:5469 write 0xb44c2cd thru 0xb45ae78 (0xebac bytes) 2024-09-16T14:09:15.739 INFO:teuthology.orchestra.run.smithi079.stdout:5470 read 0x633e452 thru 0x634cd59 (0xe908 bytes) 2024-09-16T14:09:15.739 INFO:teuthology.orchestra.run.smithi079.stdout:5472 writesame 0x2b01d8c thru 0x2b0ddbe data_size 0x63(0xc033 bytes) 2024-09-16T14:09:15.759 INFO:teuthology.orchestra.run.smithi079.stdout:5473 trunc from 0xb45ae79 to 0x2474265 2024-09-16T14:09:15.800 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:15.795+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:15.815 INFO:teuthology.orchestra.run.smithi079.stdout:5474 writesame 0xb90f3aa thru 0xb912033 data_size 0x1(0x2c8a bytes) 2024-09-16T14:09:15.817 INFO:teuthology.orchestra.run.smithi079.stdout:5476 trunc from 0xb912034 to 0xcef1c27 2024-09-16T14:09:15.826 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:15.826 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:15.832 INFO:teuthology.orchestra.run.smithi079.stdout:5477 trunc from 0xcef1c27 to 0xed8e6ba 2024-09-16T14:09:15.835 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:15.845 INFO:teuthology.orchestra.run.smithi079.stdout:5478 trunc from 0xed8e6ba to 0xd8ed91e 2024-09-16T14:09:15.855 INFO:teuthology.orchestra.run.smithi079.stdout:5479 write 0x486b657 thru 0x4874466 (0x8e10 bytes) 2024-09-16T14:09:15.863 INFO:teuthology.orchestra.run.smithi079.stdout:5480 read 0xb214c61 thru 0xb21cd25 (0x80c5 bytes) 2024-09-16T14:09:15.863 INFO:teuthology.orchestra.run.smithi079.stdout:5482 read 0x1c5098a thru 0x1c57d5d (0x73d4 bytes) 2024-09-16T14:09:15.864 INFO:teuthology.orchestra.run.smithi079.stdout:5483 write 0x355a842 thru 0x35678e0 (0xd09f bytes) 2024-09-16T14:09:15.868 INFO:teuthology.orchestra.run.smithi079.stdout:5484 writesame 0x78ca8fa thru 0x78cf180 data_size 0x3(0x4887 bytes) 2024-09-16T14:09:15.887 INFO:teuthology.orchestra.run.smithi079.stdout:5486 writesame 0xbae128 thru 0xbb9ba9 data_size 0x1(0xba82 bytes) 2024-09-16T14:09:15.906 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:15.895+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:15.919 INFO:teuthology.orchestra.run.smithi079.stdout:5487 write 0x2b6fb44 thru 0x2b72b74 (0x3031 bytes) 2024-09-16T14:09:15.929 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:15.929 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:15.938 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:15.945 INFO:teuthology.orchestra.run.smithi079.stdout:5488 trunc from 0xd8ed91e to 0x33bf1d3 2024-09-16T14:09:15.968 INFO:teuthology.orchestra.run.smithi079.stdout:5489 clone 46 order 22 su 65536 sc 8 2024-09-16T14:09:16.001 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:15.993+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:16.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:16.102 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:16.101+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:16.190 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:16.190 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:16.199 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:16.203 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:16.201+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:16.302 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:16.302 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:16.304 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:16.299+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:16.311 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:16.405 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:16.399+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:16.457 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:16.458 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:16.466 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:16.506 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:16.503+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:16.563 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:16.564 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:16.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:16.619 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:16.617+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:16.685 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:09:16.687 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:16.709 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:16.705+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:16.789 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:16.789 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:16.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:16.810 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:16.805+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:16.911 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:16.909+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:16.916 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:16.916 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:16.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:17.012 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:17.007+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:17.031 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:17.032 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:17.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:17.113 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:17.107+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:17.197 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:17.197 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:17.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:09:17.213 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:17.207+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:17.304 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:17.304 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:17.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:09:17.314 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:17.311+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:17.415 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:17.413+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:17.469 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:17.469 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:17.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:09:17.516 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:17.511+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:17.586 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:17.587 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:17.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:17.642 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:17.639+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:17.703 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:17.703 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:17.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:17.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:17.717 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:17.713+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:17.731 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone45 from 0x33bf1d3 (overlap 0x96ca23) to 0x0 2024-09-16T14:09:17.818 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:17.813+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:17.826 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:17.827 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:17.832 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:09:17.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:17.856 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:17.861 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-16T14:09:17.875 INFO:teuthology.orchestra.run.smithi079.stdout:5490 punch from 0x21be88c to 0x21cceb4, (0xe628 bytes) 2024-09-16T14:09:17.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:17.886 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:17.891 INFO:teuthology.orchestra.run.smithi079.stdout:5493 trunc from 0x33bf1d3 to 0x5b05f0b 2024-09-16T14:09:17.903 INFO:teuthology.orchestra.run.smithi079.stdout:5494 punch from 0x25384aa to 0x253b07a, (0x2bd0 bytes) 2024-09-16T14:09:17.904 INFO:teuthology.orchestra.run.smithi079.stdout:5495 clone 47 order 24 su 65536 sc 10 2024-09-16T14:09:17.905 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:17.918 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:17.915+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:17.935 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.033236290999994, 2024-09-16T14:09:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:09:17.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:09:17.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.033213583000006, 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:17.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:09:17.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.033189430999997, 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:09:17.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:17.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.033156489999996, 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.957 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:09:17.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:17.962 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.033131807000004, 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:09:17.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:09:17.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.033108577999997, 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.968 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:09:17.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:09:17.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.973 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.031262613999999, 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:09:17.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:09:17.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.031219750000005, 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:17.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029760132000007, 2024-09-16T14:09:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:09:17.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029744363000006, 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:09:17.990 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:09:17.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:09:17.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:17.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:17.995 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029728610999996, 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:17.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:09:17.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:17.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:09:18.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029713180000002, 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.001 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:09:18.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:18.006 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029697541999994, 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:18.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:09:18.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029682402999995, 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.012 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:09:18.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029667566000001, 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.017 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:18.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:09:18.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.023 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:18.023 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029644562000001, 2024-09-16T14:09:18.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:09:18.023 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.023 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:09:18.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029629337000003, 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.028 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:09:18.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:09:18.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029614152999997, 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.034 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:09:18.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:09:18.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.029598907999997, 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.039 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.02958375, 2024-09-16T14:09:18.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:09:18.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.050 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:18.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:18.052 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:18.052 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:18.052 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:09:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:18.054 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:18.049+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:18.152 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:18.149+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:18.210 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:18.211 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:18.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:18.253 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:18.249+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:18.325 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:18.325 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:18.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:18.378 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:09:18.375+0000 7f16967f4700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22181) UID: 0 2024-09-16T14:09:18.398 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 1] out_osds: [7, 3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:09:18.399 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:09:18.399 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:09:18.399 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T14:09:18.399 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:09:18.433 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.910357368, 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:09:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:09:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:09:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:09:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:09:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:09:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 36.905086029000003, 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:09:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.722577515000001, 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.719176984000001, 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:09:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.715125704999998, 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:09:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:09:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.711372788999999, 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:09:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.707293261, 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.707276659000001, 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:09:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:09:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.707260683000001, 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:09:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.707242731000001, 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:09:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.703047600000001, 2024-09-16T14:09:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.702950621999999, 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:09:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:09:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.702926081000001, 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:09:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:09:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:09:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.699369318999999, 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:09:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:09:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:09:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.699351561, 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:09:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:09:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.699333897999999, 2024-09-16T14:09:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:09:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:09:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.699316704000001, 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.695620286, 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:09:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 27.683290333999999, 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:09:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:09:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:09:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.0097781619999999, 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:09:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:09:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:18.540 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:18.542 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:18.543 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:18.537+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:18.643 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:18.639+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:18.652 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:18.652 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:18.662 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:18.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:09:18.746 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:18.743+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:18.766 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:18.767 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:18.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:18.844 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:18.839+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.041700087999999, 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.876 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:18.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:18.882 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.041683749000001, 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:18.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:18.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.041658583999997, 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.889 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:18.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.889 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:18.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:18.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:18.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.894 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.040754616999997, 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:18.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.899 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.040745278000003, 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.900 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:18.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:18.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:18.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:18.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:18.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:18.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.040455156, 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.040236462000003, 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:18.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.917 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.033842882999998, 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:18.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:18.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.923 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:18.924 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.029931320000003, 2024-09-16T14:09:18.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:18.924 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.924 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.924 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.924 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.924 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.924 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:18.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:18.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.029900114999997, 2024-09-16T14:09:18.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:18.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:18.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:18.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:18.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:18.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:18.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:18.935 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.029080675000003, 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:18.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:18.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:18.940 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.029035866000001, 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:18.941 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:18.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:18.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:18.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:18.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:18.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.026026612000003, 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:18.948 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:18.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.026009651999999, 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.954 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:18.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:18.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.025998156, 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.959 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:18.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:18.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:18.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:18.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:18.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.965 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.965 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.965 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.021251245999999, 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:18.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:18.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.970 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.013618614999999, 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.971 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:18.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:18.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.013606408000001, 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:18.976 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:18.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:18.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.981 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.011351492999999, 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:18.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:18.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:18.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:18.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:18.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:18.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:18.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:18.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:18.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:18.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:18.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:18.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:18.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:18.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:18.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.991 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:18.991 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:18.991 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.011127242000001, 2024-09-16T14:09:18.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:18.991 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:18.991 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:18.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:18.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:18.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:18.992 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:18.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.992 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:18.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:18.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:18.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.997 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:18.997 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:18.997 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:18.998 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:18.999 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:18.997+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:19.099 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:19.095+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:19.104 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3.25c07b6b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:19.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.41945685100000002, 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.41948118400000001, 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for peered", 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:09:19.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:09:19.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:09:19.109 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3.25c07b6b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.41359371700000003, 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.41366652900000001, 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for peered", 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.110 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:09:19.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 2 2024-09-16T14:09:19.114 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:19.115 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:19.200 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:19.195+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:19.217 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:19.217 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:19.217 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:19.217 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:19.217 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:19.226 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:19.301 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:19.297+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:19.329 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:19.329 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:19.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:19.329 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:19.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.329 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.951952961000003, 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:09:19.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:09:19.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:09:19.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.948021542999996, 2024-09-16T14:09:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:09:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:09:19.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:09:19.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.947997350999998, 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:09:19.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:09:19.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.944278080000004, 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:19.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:09:19.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:09:19.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:09:19.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:09:19.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:09:19.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.940361315000004, 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:09:19.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:09:19.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.935979036000006, 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.935955195000005, 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:19.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:09:19.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.935940006999999, 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:09:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:09:19.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:09:19.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.935925420999993, 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.935821770999993, 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:09:19.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:09:19.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:09:19.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.384 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.935794044999994, 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:09:19.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:09:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:09:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:09:19.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123981 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7045120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.933120428999999, 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070537616999999997, 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123981 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393284+0000", 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393285+0000", 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.39e-07 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393298+0000", 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3217000000000001e-05 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393304+0000", 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8939999999999998e-06 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393308+0000", 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0419999999999996e-06 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463701+0000", 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070392654999999998 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463759+0000", 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7948999999999997e-05 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463822+0000", 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3521000000000004e-05 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.933093714999998, 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.395 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:09:19.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.933053548000004, 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:09:19.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.932963833000002, 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:09:19.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:09:19.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:09:19.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123985 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9011200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.408 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.932555029, 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070550079000000002, 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123985 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.409 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393850+0000", 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393921+0000", 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1376000000000002e-05 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393925+0000", 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.048e-06 2024-09-16T14:09:19.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393941+0000", 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5815e-05 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393945+0000", 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2089999999999999e-06 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464289+0000", 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070343491999999994 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464340+0000", 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0733999999999999e-05 2024-09-16T14:09:19.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464400+0000", 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0405000000000002e-05 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.932454844999995, 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:09:19.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.420 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.924357474999994, 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:09:19.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:09:19.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.922189183, 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:09:19.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:09:19.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.772627721999999, 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:19.432 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:19.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:09:19.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:09:19.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.439 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:19.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:19.440 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:19.437+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:19.541 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:19.537+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:19.589 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:19.589 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:19.589 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:19.589 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:19.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:19.642 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:19.637+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:19.703 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:09:19.709 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:19.762 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:19.759+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:19.843 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:19.839+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:19.857 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone46 from 0x5b05f0b (overlap 0x33bf1d3) to 0x2438fad 2024-09-16T14:09:19.860 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:19.860 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:19.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:19.860 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 118.445593765, 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:19.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:19.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:19.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:19.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:19.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 90.607631441999999, 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:19.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:19.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:19.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:19.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.080392453999998, 2024-09-16T14:09:19.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:19.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.080369222999998, 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.879 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:19.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.883 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.080346276, 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.884 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:19.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.888 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.080329767000002, 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:19.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.893 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.080312141, 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:19.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:19.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.074396397000001, 2024-09-16T14:09:19.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:19.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.074378992, 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:19.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:19.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.074024626000003, 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.908 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:19.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:19.912 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.070880766999998, 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.913 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:19.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.070865480999998, 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:19.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.922 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.070849072000001, 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:19.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.070773875, 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.927 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.931 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.070676327999998, 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.932 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.070651087000002, 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.070622032999999, 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.067358044000002, 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.067339842999999, 2024-09-16T14:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.067300664999998, 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:19.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:19.965 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:19.961+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:20.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:20.064 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:20.059+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:20.070 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-16T14:09:20.073 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:20.073 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:20.073 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:20.073 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:20.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:20.097 INFO:teuthology.orchestra.run.smithi079.stdout:5498 trunc from 0x5b05f0b to 0x17cbdf5 2024-09-16T14:09:20.156 INFO:teuthology.orchestra.run.smithi079.stdout:5500 punch from 0xebb042 to 0xec2b9c, (0x7b5a bytes) 2024-09-16T14:09:20.156 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.156 INFO:teuthology.orchestra.run.smithi079.stdout:5502 punch from 0x11379d4 to 0x113f77e, (0x7daa bytes) 2024-09-16T14:09:20.163 INFO:teuthology.orchestra.run.smithi079.stdout:5503 trunc from 0x17cbdf5 to 0x5f0a6f9 2024-09-16T14:09:20.165 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:20.161+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:20.171 INFO:teuthology.orchestra.run.smithi079.stdout:5504 read 0x59916b8 thru 0x59995df (0x7f28 bytes) 2024-09-16T14:09:20.185 INFO:teuthology.orchestra.run.smithi079.stdout:5505 write 0x8ac05a9 thru 0x8ac9350 (0x8da8 bytes) 2024-09-16T14:09:20.187 INFO:teuthology.orchestra.run.smithi079.stdout:5506 punch from 0x3dbb8f8 to 0x3dc1d3d, (0x6445 bytes) 2024-09-16T14:09:20.199 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:20.199 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:20.199 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:20.199 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:20.199 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:20.200 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:20.202 INFO:teuthology.orchestra.run.smithi079.stdout:5507 write 0xbef8acc thru 0xbeff06d (0x65a2 bytes) 2024-09-16T14:09:20.207 INFO:teuthology.orchestra.run.smithi079.stdout:5508 read 0xb8ca7c thru 0xb8f4d4 (0x2a59 bytes) 2024-09-16T14:09:20.208 INFO:teuthology.orchestra.run.smithi079.stdout:5510 read 0x47798b6 thru 0x4787678 (0xddc3 bytes) 2024-09-16T14:09:20.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:20.209 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.221 INFO:teuthology.orchestra.run.smithi079.stdout:5512 write 0xcf69eda thru 0xcf74b49 (0xac70 bytes) 2024-09-16T14:09:20.223 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:20.227 INFO:teuthology.orchestra.run.smithi079.stdout:5513 write 0xcb682e thru 0xcc135f (0xab32 bytes) 2024-09-16T14:09:20.230 INFO:teuthology.orchestra.run.smithi079.stdout:5514 write 0x13d60a2 thru 0x13e5019 (0xef78 bytes) 2024-09-16T14:09:20.234 INFO:teuthology.orchestra.run.smithi079.stdout:5515 punch from 0x147d52e to 0x1485bf0, (0x86c2 bytes) 2024-09-16T14:09:20.234 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.234 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.245 INFO:teuthology.orchestra.run.smithi079.stdout:5520 write 0xe29ecca thru 0xe2a4338 (0x566f bytes) 2024-09-16T14:09:20.245 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:20.263 INFO:teuthology.orchestra.run.smithi079.stdout:5521 write 0xe76321f thru 0xe76f977 (0xc759 bytes) 2024-09-16T14:09:20.266 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:20.263+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout:5522 read 0x5ae1da2 thru 0x5aeac75 (0x8ed4 bytes) 2024-09-16T14:09:20.271 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:20.272 INFO:teuthology.orchestra.run.smithi079.stdout:5523 read 0x59d4017 thru 0x59e145e (0xd448 bytes) 2024-09-16T14:09:20.283 INFO:teuthology.orchestra.run.smithi079.stdout:5524 trunc from 0xe76f978 to 0x91272be 2024-09-16T14:09:20.295 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:20.307 INFO:teuthology.orchestra.run.smithi079.stdout:5525 punch from 0x3f7e907 to 0x3f880a3, (0x979c bytes) 2024-09-16T14:09:20.317 INFO:teuthology.orchestra.run.smithi079.stdout:5526 trunc from 0x91272be to 0x3056f15 2024-09-16T14:09:20.329 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:20.335 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:20.336 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:20.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:20.354 INFO:teuthology.orchestra.run.smithi079.stdout:5528 write 0x9849b3d thru 0x984b98f (0x1e53 bytes) 2024-09-16T14:09:20.357 INFO:teuthology.orchestra.run.smithi079.stdout:5530 writesame 0x318ca99 thru 0x319b2ca data_size 0x3(0xe832 bytes) 2024-09-16T14:09:20.359 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:20.359 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:20.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:20.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:20.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:20.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:20.360 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:20.360 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:20.360 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:20.363 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:20.363 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:20.364 INFO:teuthology.orchestra.run.smithi079.stdout:5531 punch from 0x4d2fc79 to 0x4d31903, (0x1c8a bytes) 2024-09-16T14:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout:5532 read 0x2c375e6 thru 0x2c3db6c (0x6587 bytes) 2024-09-16T14:09:20.365 INFO:teuthology.orchestra.run.smithi079.stdout:5534 write 0x233f3c9 thru 0x2340a85 (0x16bd bytes) 2024-09-16T14:09:20.368 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:20.365+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:20.368 INFO:teuthology.orchestra.run.smithi079.stdout:5535 writesame 0x72ebf56 thru 0x72ec57d data_size 0x4(0x628 bytes) 2024-09-16T14:09:20.370 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.370 INFO:teuthology.orchestra.run.smithi079.stdout:5538 write 0x51a9dec thru 0x51ae36b (0x4580 bytes) 2024-09-16T14:09:20.372 INFO:teuthology.orchestra.run.smithi079.stdout:5541 write 0x5f48339 thru 0x5f5358e (0xb256 bytes) 2024-09-16T14:09:20.376 INFO:teuthology.orchestra.run.smithi079.stdout:5542 read 0x61dfa29 thru 0x61ed07e (0xd656 bytes) 2024-09-16T14:09:20.381 INFO:teuthology.orchestra.run.smithi079.stdout:5544 trunc from 0x984b990 to 0xc383ac6 2024-09-16T14:09:20.391 INFO:teuthology.orchestra.run.smithi079.stdout:5545 read 0x53b806 thru 0x5404f4 (0x4cef bytes) 2024-09-16T14:09:20.392 INFO:teuthology.orchestra.run.smithi079.stdout:5546 write 0x5244d9f thru 0x52456ce (0x930 bytes) 2024-09-16T14:09:20.394 INFO:teuthology.orchestra.run.smithi079.stdout:5548 writesame 0x5392bce thru 0x53a2040 data_size 0x1(0xf473 bytes) 2024-09-16T14:09:20.443 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.453 INFO:teuthology.orchestra.run.smithi079.stdout:5551 trunc from 0xc383ac6 to 0x669a59f 2024-09-16T14:09:20.456 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:20.456 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:20.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:20.468 INFO:teuthology.orchestra.run.smithi079.stdout:5552 punch from 0x4f417a1 to 0x4f46919, (0x5178 bytes) 2024-09-16T14:09:20.468 INFO:teuthology.orchestra.run.smithi079.stdout:5554 read 0x58702a thru 0x58cbd0 (0x5ba7 bytes) 2024-09-16T14:09:20.468 INFO:teuthology.orchestra.run.smithi079.stdout:5555 write 0x2768908 thru 0x276901d (0x716 bytes) 2024-09-16T14:09:20.469 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:20.465+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:20.480 INFO:teuthology.orchestra.run.smithi079.stdout:5557 trunc from 0x669a59f to 0xd287619 2024-09-16T14:09:20.491 INFO:teuthology.orchestra.run.smithi079.stdout:5558 write 0xad1ddf9 thru 0xad26f01 (0x9109 bytes) 2024-09-16T14:09:20.499 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.499 INFO:teuthology.orchestra.run.smithi079.stdout:5561 writesame 0x438da46 thru 0x43913dd data_size 0x8(0x3998 bytes) 2024-09-16T14:09:20.504 INFO:teuthology.orchestra.run.smithi079.stdout:5562 read 0x9368a8f thru 0x9371eb7 (0x9429 bytes) 2024-09-16T14:09:20.505 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.516 INFO:teuthology.orchestra.run.smithi079.stdout:5567 write 0xd9bea0c thru 0xd9c8bb7 (0xa1ac bytes) 2024-09-16T14:09:20.525 INFO:teuthology.orchestra.run.smithi079.stdout:5570 punch from 0x11411d0 to 0x1147ea6, (0x6cd6 bytes) 2024-09-16T14:09:20.525 INFO:teuthology.orchestra.run.smithi079.stdout:5571 punch from 0xca5bdd2 to 0xca5c485, (0x6b3 bytes) 2024-09-16T14:09:20.525 INFO:teuthology.orchestra.run.smithi079.stdout:5573 read 0x11f5f9b thru 0x11fda74 (0x7ada bytes) 2024-09-16T14:09:20.526 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:20.526 INFO:teuthology.orchestra.run.smithi079.stdout:5577 write 0x84ee523 thru 0x84fbdf3 (0xd8d1 bytes) 2024-09-16T14:09:20.529 INFO:teuthology.orchestra.run.smithi079.stdout:5579 read 0x5c46e3 thru 0x5c6ac6 (0x23e4 bytes) 2024-09-16T14:09:20.530 INFO:teuthology.orchestra.run.smithi079.stdout:5580 write 0x8ee92f7 thru 0x8ef4933 (0xb63d bytes) 2024-09-16T14:09:20.534 INFO:teuthology.orchestra.run.smithi079.stdout:5581 writesame 0x4599ad5 thru 0x45a0f26 data_size 0x15(0x7452 bytes) 2024-09-16T14:09:20.538 INFO:teuthology.orchestra.run.smithi079.stdout:5583 punch from 0x89cefb9 to 0x89d96d0, (0xa717 bytes) 2024-09-16T14:09:20.550 INFO:teuthology.orchestra.run.smithi079.stdout:5588 trunc from 0xd9c8bb8 to 0x68266f1 2024-09-16T14:09:20.570 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:20.565+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:20.572 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:20.572 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:20.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:20.588 INFO:teuthology.orchestra.run.smithi079.stdout:5591 write 0x8b77ccc thru 0x8b7f254 (0x7589 bytes) 2024-09-16T14:09:20.592 INFO:teuthology.orchestra.run.smithi079.stdout:5592 writesame 0x796cf16 thru 0x79770b1 data_size 0x2(0xa19c bytes) 2024-09-16T14:09:20.601 INFO:teuthology.orchestra.run.smithi079.stdout:5593 write 0x1e356 thru 0x21f4d (0x3bf8 bytes) 2024-09-16T14:09:20.604 INFO:teuthology.orchestra.run.smithi079.stdout:5594 punch from 0x2bb9082 to 0x2bbe109, (0x5087 bytes) 2024-09-16T14:09:20.613 INFO:teuthology.orchestra.run.smithi079.stdout:5595 trunc from 0x8b7f255 to 0x2defbf2 2024-09-16T14:09:20.631 INFO:teuthology.orchestra.run.smithi079.stdout:5596 clone 48 order 22 su 262144 sc 5 2024-09-16T14:09:20.671 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:20.667+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:20.687 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:20.687 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:20.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:20.771 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:20.769+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:20.846 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:20.846 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:20.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:20.872 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:20.869+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:20.896 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone47 from 0x2defbf2 (overlap 0x17cbdf5) to 0x984bfb 2024-09-16T14:09:20.961 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:20.961 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:20.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:20.973 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:20.969+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:21.069 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.069 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.073 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:21.067+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:21.082 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:21.140 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-16T14:09:21.175 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:21.171+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.214 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:21.250 INFO:teuthology.orchestra.run.smithi079.stdout:5597 read 0x1d3fead thru 0x1d4d1b8 (0xd30c bytes) 2024-09-16T14:09:21.276 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:21.273+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:21.281 INFO:teuthology.orchestra.run.smithi079.stdout:5598 write 0x7153eaa thru 0x7160b4f (0xcca6 bytes) 2024-09-16T14:09:21.286 INFO:teuthology.orchestra.run.smithi079.stdout:5600 read 0x6d63060 thru 0x6d71433 (0xe3d4 bytes) 2024-09-16T14:09:21.286 INFO:teuthology.orchestra.run.smithi079.stdout:5601 writesame 0x3ccbca6 thru 0x3cd07a2 data_size 0x1b(0x4afd bytes) 2024-09-16T14:09:21.290 INFO:teuthology.orchestra.run.smithi079.stdout:5602 read 0x6ad5bba thru 0x6ada078 (0x44bf bytes) 2024-09-16T14:09:21.291 INFO:teuthology.orchestra.run.smithi079.stdout:5603 punch from 0x374b999 to 0x37566df, (0xad46 bytes) 2024-09-16T14:09:21.291 INFO:teuthology.orchestra.run.smithi079.stdout:5604 write 0x59fbbcc thru 0x5a0a4a9 (0xe8de bytes) 2024-09-16T14:09:21.298 INFO:teuthology.orchestra.run.smithi079.stdout:5605 write 0x2760aa4 thru 0x276fbc4 (0xf121 bytes) 2024-09-16T14:09:21.302 INFO:teuthology.orchestra.run.smithi079.stdout:5608 write 0x152ef9a thru 0x15337c4 (0x482b bytes) 2024-09-16T14:09:21.312 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.312 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.316 INFO:teuthology.orchestra.run.smithi079.stdout:5609 write 0xe740aa thru 0xe77b1a (0x3a71 bytes) 2024-09-16T14:09:21.322 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:21.333 INFO:teuthology.orchestra.run.smithi079.stdout:5610 trunc from 0x7160b50 to 0x46b838f 2024-09-16T14:09:21.354 INFO:teuthology.orchestra.run.smithi079.stdout:5611 read 0x27338fd thru 0x273ea7c (0xb180 bytes) 2024-09-16T14:09:21.377 INFO:teuthology.orchestra.run.smithi079.stdout:5612 write 0xaf8c400 thru 0xaf909e6 (0x45e7 bytes) 2024-09-16T14:09:21.377 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:21.371+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:21.384 INFO:teuthology.orchestra.run.smithi079.stdout:5614 punch from 0x363cfeb to 0x363dd03, (0xd18 bytes) 2024-09-16T14:09:21.384 INFO:teuthology.orchestra.run.smithi079.stdout:5615 writesame 0x6ca9928 thru 0x6cb7d37 data_size 0xa4(0xe410 bytes) 2024-09-16T14:09:21.390 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:21.390 INFO:teuthology.orchestra.run.smithi079.stdout:5617 read 0x814bb61 thru 0x814c691 (0xb31 bytes) 2024-09-16T14:09:21.397 INFO:teuthology.orchestra.run.smithi079.stdout:5619 trunc from 0xaf909e7 to 0x6ff5bce 2024-09-16T14:09:21.417 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.417 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.427 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:21.440 INFO:teuthology.orchestra.run.smithi079.stdout:5620 writesame 0xb53c566 thru 0xb540c7c data_size 0x1(0x4717 bytes) 2024-09-16T14:09:21.450 INFO:teuthology.orchestra.run.smithi079.stdout:5622 trunc from 0xb540c7d to 0x6d92e87 2024-09-16T14:09:21.466 INFO:teuthology.orchestra.run.smithi079.stdout:5623 trunc from 0x6d92e87 to 0x71fe0c5 2024-09-16T14:09:21.478 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:21.475+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:21.484 INFO:teuthology.orchestra.run.smithi079.stdout:5624 trunc from 0x71fe0c5 to 0x39f14c4 2024-09-16T14:09:21.519 INFO:teuthology.orchestra.run.smithi079.stdout:5625 read 0x6dc529 thru 0x6e90c3 (0xcb9b bytes) 2024-09-16T14:09:21.520 INFO:teuthology.orchestra.run.smithi079.stdout:5627 read 0x86a6d6 thru 0x8789d6 (0xe301 bytes) 2024-09-16T14:09:21.521 INFO:teuthology.orchestra.run.smithi079.stdout:5628 read 0x1f48a2c thru 0x1f58680 (0xfc55 bytes) 2024-09-16T14:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout:5629 punch from 0x2801111 to 0x280bdf4, (0xace3 bytes) 2024-09-16T14:09:21.525 INFO:teuthology.orchestra.run.smithi079.stdout:5630 read 0x2790d70 thru 0x27966de (0x596f bytes) 2024-09-16T14:09:21.529 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.529 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.539 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:21.550 INFO:teuthology.orchestra.run.smithi079.stdout:5631 trunc from 0x39f14c4 to 0xdd41067 2024-09-16T14:09:21.556 INFO:teuthology.orchestra.run.smithi079.stdout:5632 read 0x6fc1b6b thru 0x6fcaa3c (0x8ed2 bytes) 2024-09-16T14:09:21.556 INFO:teuthology.orchestra.run.smithi079.stdout:5634 read 0x8f851c3 thru 0x8f9105f (0xbe9d bytes) 2024-09-16T14:09:21.556 INFO:teuthology.orchestra.run.smithi079.stdout:5635 read 0x9fe7dc1 thru 0x9fef3b0 (0x75f0 bytes) 2024-09-16T14:09:21.556 INFO:teuthology.orchestra.run.smithi079.stdout:5636 writesame 0xb8dac4e thru 0xb8daf2e data_size 0xb(0x2e1 bytes) 2024-09-16T14:09:21.559 INFO:teuthology.orchestra.run.smithi079.stdout:5640 write 0xe43b4a thru 0xe51b42 (0xdff9 bytes) 2024-09-16T14:09:21.563 INFO:teuthology.orchestra.run.smithi079.stdout:5642 write 0x1d4caf3 thru 0x1d54a46 (0x7f54 bytes) 2024-09-16T14:09:21.566 INFO:teuthology.orchestra.run.smithi079.stdout:5643 writesame 0x94cf927 thru 0x94d3016 data_size 0x18(0x36f0 bytes) 2024-09-16T14:09:21.570 INFO:teuthology.orchestra.run.smithi079.stdout:5644 writesame 0xb9ec626 thru 0xb9f26d4 data_size 0x1(0x60af bytes) 2024-09-16T14:09:21.579 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:21.575+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:21.599 INFO:teuthology.orchestra.run.smithi079.stdout:5645 trunc from 0xdd41067 to 0x542d68b 2024-09-16T14:09:21.612 INFO:teuthology.orchestra.run.smithi079.stdout:5646 read 0x2af10f9 thru 0x2af14f7 (0x3ff bytes) 2024-09-16T14:09:21.631 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.631 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.633 INFO:teuthology.orchestra.run.smithi079.stdout:5647 write 0xce2f8f7 thru 0xce340a3 (0x47ad bytes) 2024-09-16T14:09:21.640 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:21.641 INFO:teuthology.orchestra.run.smithi079.stdout:5649 read 0xa97ad3c thru 0xa988450 (0xd715 bytes) 2024-09-16T14:09:21.641 INFO:teuthology.orchestra.run.smithi079.stdout:5650 read 0xa5277f thru 0xa5d8d8 (0xb15a bytes) 2024-09-16T14:09:21.642 INFO:teuthology.orchestra.run.smithi079.stdout:5651 read 0x1035bdb thru 0x103ef14 (0x933a bytes) 2024-09-16T14:09:21.643 INFO:teuthology.orchestra.run.smithi079.stdout:5652 writesame 0xb2b7e25 thru 0xb2bb754 data_size 0x50(0x3930 bytes) 2024-09-16T14:09:21.678 INFO:teuthology.orchestra.run.smithi079.stdout:5653 trunc from 0xce340a4 to 0x322f945 2024-09-16T14:09:21.687 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:21.685+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:21.697 INFO:teuthology.orchestra.run.smithi079.stdout:5654 read 0x188180 thru 0x194dc0 (0xcc41 bytes) 2024-09-16T14:09:21.698 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:21.698 INFO:teuthology.orchestra.run.smithi079.stdout:5657 punch from 0x11d54f to 0x12362e, (0x60df bytes) 2024-09-16T14:09:21.698 INFO:teuthology.orchestra.run.smithi079.stdout:5660 read 0x1f20a6d thru 0x1f2cac1 (0xc055 bytes) 2024-09-16T14:09:21.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:09:21.725 INFO:teuthology.orchestra.run.smithi079.stdout:5661 writesame 0xb253804 thru 0xb25a2b3 data_size 0x18(0x6ab0 bytes) 2024-09-16T14:09:21.725 INFO:teuthology.orchestra.run.smithi079.stdout:5662 read 0x57c46e8 thru 0x57c500c (0x925 bytes) 2024-09-16T14:09:21.725 INFO:teuthology.orchestra.run.smithi079.stdout:5663 write 0x45aa41 thru 0x4627e2 (0x7da2 bytes) 2024-09-16T14:09:21.728 INFO:teuthology.orchestra.run.smithi079.stdout:5664 read 0x1792409 thru 0x179703d (0x4c35 bytes) 2024-09-16T14:09:21.729 INFO:teuthology.orchestra.run.smithi079.stdout:5665 read 0x785ef42 thru 0x786856c (0x962b bytes) 2024-09-16T14:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.739 INFO:teuthology.orchestra.run.smithi079.stdout:5666 trunc from 0xb25a2b4 to 0x68a76f0 2024-09-16T14:09:21.743 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:21.746 INFO:teuthology.orchestra.run.smithi079.stdout:5667 punch from 0x65e31b4 to 0x65e86d1, (0x551d bytes) 2024-09-16T14:09:21.747 INFO:teuthology.orchestra.run.smithi079.stdout:5669 write 0x4b252f9 thru 0x4b2a60a (0x5312 bytes) 2024-09-16T14:09:21.751 INFO:teuthology.orchestra.run.smithi079.stdout:5670 punch from 0x1fe1bf1 to 0x1fe92b4, (0x76c3 bytes) 2024-09-16T14:09:21.774 INFO:teuthology.orchestra.run.smithi079.stdout:5671 write 0xda0807d thru 0xda1774f (0xf6d3 bytes) 2024-09-16T14:09:21.779 INFO:teuthology.orchestra.run.smithi079.stdout:5672 write 0xb4de678 thru 0xb4e82e6 (0x9c6f bytes) 2024-09-16T14:09:21.780 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:21.777+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:21.784 INFO:teuthology.orchestra.run.smithi079.stdout:5673 writesame 0xbdeda8 thru 0xbe40a4 data_size 0x7(0x52fd bytes) 2024-09-16T14:09:21.789 INFO:teuthology.orchestra.run.smithi079.stdout:5674 read 0xceb7a6e thru 0xcebe040 (0x65d3 bytes) 2024-09-16T14:09:21.793 INFO:teuthology.orchestra.run.smithi079.stdout:5675 trunc from 0xda17750 to 0xb7574c1 2024-09-16T14:09:21.812 INFO:teuthology.orchestra.run.smithi079.stdout:5676 write 0x997bfe7 thru 0x998856d (0xc587 bytes) 2024-09-16T14:09:21.820 INFO:teuthology.orchestra.run.smithi079.stdout:5677 read 0x9b7ecc9 thru 0x9b8257a (0x38b2 bytes) 2024-09-16T14:09:21.824 INFO:teuthology.orchestra.run.smithi079.stdout:5679 trunc from 0xb7574c1 to 0xd588b12 2024-09-16T14:09:21.831 INFO:teuthology.orchestra.run.smithi079.stdout:5680 write 0x875ed4e thru 0x87694bd (0xa770 bytes) 2024-09-16T14:09:21.836 INFO:teuthology.orchestra.run.smithi079.stdout:5681 writesame 0x32a7865 thru 0x32ad55f data_size 0x1(0x5cfb bytes) 2024-09-16T14:09:21.840 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.840 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.847 INFO:teuthology.orchestra.run.smithi079.stdout:5682 write 0x6ca5614 thru 0x6cad970 (0x835d bytes) 2024-09-16T14:09:21.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:21.854 INFO:teuthology.orchestra.run.smithi079.stdout:5684 read 0x491bfa2 thru 0x49278e0 (0xb93f bytes) 2024-09-16T14:09:21.854 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:21.854 INFO:teuthology.orchestra.run.smithi079.stdout:5686 punch from 0xb48913c to 0xb497b16, (0xe9da bytes) 2024-09-16T14:09:21.854 INFO:teuthology.orchestra.run.smithi079.stdout:5687 read 0x1643a75 thru 0x16475c9 (0x3b55 bytes) 2024-09-16T14:09:21.855 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:21.855 INFO:teuthology.orchestra.run.smithi079.stdout:5690 read 0x504de9c thru 0x5050132 (0x2297 bytes) 2024-09-16T14:09:21.873 INFO:teuthology.orchestra.run.smithi079.stdout:5693 trunc from 0xd588b12 to 0x2385c5c 2024-09-16T14:09:21.898 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:21.897+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:21.903 INFO:teuthology.orchestra.run.smithi079.stdout:5694 writesame 0x3932d0f thru 0x393d668 data_size 0x35(0xa95a bytes) 2024-09-16T14:09:21.904 INFO:teuthology.orchestra.run.smithi079.stdout:5695 read 0x32a04fa thru 0x32a9839 (0x9340 bytes) 2024-09-16T14:09:21.904 INFO:teuthology.orchestra.run.smithi079.stdout:5696 read 0x746646 thru 0x749c8b (0x3646 bytes) 2024-09-16T14:09:21.904 INFO:teuthology.orchestra.run.smithi079.stdout:5697 write 0x1051c2 thru 0x1096a2 (0x44e1 bytes) 2024-09-16T14:09:21.911 INFO:teuthology.orchestra.run.smithi079.stdout:5698 trunc from 0x393d669 to 0xa1fd48 2024-09-16T14:09:21.941 INFO:teuthology.orchestra.run.smithi079.stdout:5699 read 0xf165a thru 0xfcf00 (0xb8a7 bytes) 2024-09-16T14:09:21.952 INFO:teuthology.orchestra.run.smithi079.stdout:5700 write 0x15d86e3 thru 0x15e4e7b (0xc799 bytes) 2024-09-16T14:09:21.953 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:21.954 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:21.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:21.971 INFO:teuthology.orchestra.run.smithi079.stdout:5702 trunc from 0x15e4e7c to 0xa7ddf62 2024-09-16T14:09:21.978 INFO:teuthology.orchestra.run.smithi079.stdout:5703 write 0x30e19fa thru 0x30e901f (0x7626 bytes) 2024-09-16T14:09:21.983 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:21.979+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:21.983 INFO:teuthology.orchestra.run.smithi079.stdout:5704 punch from 0x28c1c6e to 0x28d038a, (0xe71c bytes) 2024-09-16T14:09:22.001 INFO:teuthology.orchestra.run.smithi079.stdout:5705 write 0xd8e4a03 thru 0xd8f226c (0xd86a bytes) 2024-09-16T14:09:22.001 INFO:teuthology.orchestra.run.smithi079.stdout:5708 write 0x98bea0 thru 0x998207 (0xc368 bytes) 2024-09-16T14:09:22.004 INFO:teuthology.orchestra.run.smithi079.stdout:5709 punch from 0x34cee12 to 0x34d8510, (0x96fe bytes) 2024-09-16T14:09:22.004 INFO:teuthology.orchestra.run.smithi079.stdout:5711 write 0xa98d2b7 thru 0xa997730 (0xa47a bytes) 2024-09-16T14:09:22.009 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:22.009 INFO:teuthology.orchestra.run.smithi079.stdout:5714 clone 49 order 19 su 65536 sc 6 2024-09-16T14:09:22.062 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:22.063 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:22.072 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:22.083 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:22.079+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:22.170 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:22.170 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:22.180 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:22.183 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:22.179+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:22.274 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:22.275 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:22.284 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:22.284 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:22.281+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:22.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:22.379 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:22.379 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:22.384 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:22.381+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:22.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:22.485 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:22.479+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:22.508 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:22.518 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:22.519 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:22.524 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:22.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:22.543 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:22.563 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:22.586 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:22.581+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:22.586 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:22.612 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:22.625 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:22.625 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:22.625 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:22.631 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:22.631 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:22.631 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:22.631 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:22.631 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:22.632 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:22.632 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:22.632 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:22.632 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:22.635 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:22.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:22.636 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:22.687 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:22.685+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:22.739 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:09:22.787 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.547306128999999, 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187153086000002, 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187144087, 2024-09-16T14:09:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187135736000002, 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187103051999998, 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187093814999997, 2024-09-16T14:09:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187085283000002, 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187076705999999, 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187067345000003, 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187059232999999, 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187047530000001, 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187038917000002, 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187030811, 2024-09-16T14:09:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187022468999999, 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187013931000003, 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.187005599000003, 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.186997357000003, 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.186987793999997, 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.186978412999999, 2024-09-16T14:09:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.186968772, 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:22.886 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:22.887 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:22.783+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:22.887 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:22.889 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:22.885+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:22.937 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone48 from 0xd8f226d (overlap 0xa1fd48) to 0x0 2024-09-16T14:09:22.989 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:22.987+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:22.999 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:189324 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e133)", 2024-09-16T14:09:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:22.998509+0000", 2024-09-16T14:09:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00011375299999999999, 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013390200000000001, 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "queued for pg", 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 189324 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:22.998509+0000", 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:22.998509+0000", 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:22.998511+0000", 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.418e-06 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:22.998518+0000", 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.182e-06 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:22.998538+0000", 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9823e-05 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:22.998546+0000", 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4959999999999996e-06 2024-09-16T14:09:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 1 2024-09-16T14:09:23.003 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:23.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:23.090 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:23.087+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:23.168 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:23.168 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:23.168 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:23.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:23.191 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:23.189+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:23.196 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-16T14:09:23.234 INFO:teuthology.orchestra.run.smithi079.stdout:5715 read 0xa5d825a thru 0xa5dda71 (0x5818 bytes) 2024-09-16T14:09:23.245 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:23.245 INFO:teuthology.orchestra.run.smithi079.stdout:5719 punch from 0x8e25d51 to 0x8e30a52, (0xad01 bytes) 2024-09-16T14:09:23.245 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:23.245 INFO:teuthology.orchestra.run.smithi079.stdout:5721 punch from 0x7ca87b to 0x7d2b6e, (0x82f3 bytes) 2024-09-16T14:09:23.245 INFO:teuthology.orchestra.run.smithi079.stdout:5722 write 0xe1015f thru 0xe15e2d (0x5ccf bytes) 2024-09-16T14:09:23.254 INFO:teuthology.orchestra.run.smithi079.stdout:5723 punch from 0x4e272b to 0x4e4bce, (0x24a3 bytes) 2024-09-16T14:09:23.254 INFO:teuthology.orchestra.run.smithi079.stdout:5724 read 0x517da4a thru 0x5182654 (0x4c0b bytes) 2024-09-16T14:09:23.254 INFO:teuthology.orchestra.run.smithi079.stdout:5726 read 0x35e87a5 thru 0x35f4793 (0xbfef bytes) 2024-09-16T14:09:23.278 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:23.278 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:23.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:23.278 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:23.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.372461853000004, 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:09:23.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.372439145000001, 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:09:23.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:23.289 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.372414993000007, 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:09:23.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:09:23.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.372382052000006, 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:23.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:23.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:09:23.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.372357368999999, 2024-09-16T14:09:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:09:23.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:09:23.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:09:23.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:09:23.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:09:23.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:09:23.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.306 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.372334140000007, 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.307 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:09:23.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.370488175999995, 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:09:23.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:09:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.370445312000001, 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:09:23.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:09:23.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368985694000003, 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:09:23.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:09:23.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368969925000002, 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:09:23.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.334 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368954173000006, 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.339 INFO:teuthology.orchestra.run.smithi079.stdout:5727 write 0xe7a14dc thru 0xe7a56f2 (0x4217 bytes) 2024-09-16T14:09:23.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:09:23.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368938741999997, 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:23.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:09:23.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.346 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368923104000004, 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:23.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:09:23.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368907965000005, 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:09:23.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368893127999996, 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:09:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368870123999997, 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:09:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368854898999999, 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:09:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:09:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368839715000007, 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:09:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368824470000007, 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:09:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:09:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:09:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.368809311999996, 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:09:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.391 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:23.392 INFO:teuthology.orchestra.run.smithi079.stdout:5729 read 0x1f39f95 thru 0x1f41103 (0x716f bytes) 2024-09-16T14:09:23.392 INFO:teuthology.orchestra.run.smithi079.stdout:5730 write 0xcfaf80 thru 0xd078ec (0xc96d bytes) 2024-09-16T14:09:23.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:23.393 INFO:teuthology.orchestra.run.smithi079.stdout:5731 clone 50 order 19 su 131072 sc 11 2024-09-16T14:09:23.394 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:23.393+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:23.495 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:23.493+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:23.537 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:23.538 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:23.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:23.596 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:23.593+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:23.647 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:23.647 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:23.647 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:23.647 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:23.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:23.697 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:23.693+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:23.761 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:23.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.239870338000003, 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:09:23.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:09:23.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.234598998999999, 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:09:23.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:09:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:09:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:09:23.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:09:23.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:23.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:09:23.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:09:23.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:23.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:09:23.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:09:23.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:09:23.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:09:23.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.776 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.777 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.777 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:23.777 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.052090485000001, 2024-09-16T14:09:23.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:09:23.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:09:23.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.778 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:23.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:23.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:09:23.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:09:23.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:09:23.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:09:23.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:09:23.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:09:23.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.048689953999997, 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:09:23.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:09:23.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:09:23.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:09:23.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:09:23.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.789 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.044638675000002, 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:09:23.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:09:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.040885758999998, 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:09:23.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:09:23.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:09:23.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:09:23.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.036806231, 2024-09-16T14:09:23.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:23.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:23.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:23.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:09:23.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:09:23.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:09:23.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:09:23.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:09:23.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:09:23.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:09:23.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:09:23.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:09:23.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:09:23.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:09:23.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:09:23.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:09:23.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.807 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.807 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.808 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.808 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:23.808 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.036789628999998, 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:23.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:09:23.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.036773652999997, 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:09:23.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.818 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.036755700999997, 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:09:23.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.032560570000001, 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:23.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:23.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:09:23.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:09:23.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:09:23.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.828 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.032463591999999, 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:09:23.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:09:23.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.032439050999997, 2024-09-16T14:09:23.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:23.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:23.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:09:23.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:09:23.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.028882289000002, 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:09:23.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:09:23.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:09:23.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.028864531000004, 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:09:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.028846868000002, 2024-09-16T14:09:23.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:23.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:09:23.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:23.856 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.028829674000001, 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:09:23.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.025133255999997, 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:09:23.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:09:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.012803304000002, 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:09:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:09:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.3392911319999996, 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:23.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:09:23.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:09:23.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:23.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:09:23.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:23.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:23.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:09:23.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:09:23.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:23.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:09:23.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:23.879 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:23.881 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:23.881 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:23.875+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:23.982 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:23.979+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:23.996 INFO:teuthology.orchestra.run.smithi079.stdout:truncating image image_client.0-clone49 from 0xe7a56f3 (overlap 0xd8f226d) to 0x2b63a15 2024-09-16T14:09:23.997 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:189770 3.1b 3:df512b00:::rbd_object_map.1136a7a33106:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-16T14:09:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:23.995880+0000", 2024-09-16T14:09:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00083559300000000003, 2024-09-16T14:09:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00088682099999999999, 2024-09-16T14:09:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 189770 2024-09-16T14:09:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:23.995880+0000", 2024-09-16T14:09:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:23.995880+0000", 2024-09-16T14:09:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:23.995882+0000", 2024-09-16T14:09:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.39e-06 2024-09-16T14:09:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:23.995894+0000", 2024-09-16T14:09:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2130000000000001e-05 2024-09-16T14:09:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:23.995925+0000", 2024-09-16T14:09:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0895999999999997e-05 2024-09-16T14:09:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:23.995951+0000", 2024-09-16T14:09:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5389999999999999e-05 2024-09-16T14:09:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:23.996034+0000", 2024-09-16T14:09:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3441000000000003e-05 2024-09-16T14:09:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:23.996150+0000", 2024-09-16T14:09:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011581 2024-09-16T14:09:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:09:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T14:09:24.009 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:24.010 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:24.083 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:24.081+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:24.125 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:24.125 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:24.125 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:24.125 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:24.125 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:24.139 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:24.184 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:24.179+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:24.258 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.91774843499999, 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:09:24.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:24.265 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.917726038, 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:09:24.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:09:24.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:09:24.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.91711932699999, 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:24.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:24.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:09:24.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.917045, 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.279 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.946120208, 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:09:24.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:09:24.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:09:24.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 113.31816162699999, 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:09:24.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.788001161, 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:09:24.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:09:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:09:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:09:24.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:09:24.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:24.304 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.140452629999999, 2024-09-16T14:09:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:09:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:09:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:24.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:09:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:09:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:09:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:09:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:09:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:09:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:09:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:09:24.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:24.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.594087505999994, 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:24.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:09:24.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.594045326, 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.317 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:09:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:09:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:09:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:09:24.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:09:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:09:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:09:24.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:09:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:09:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:09:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:09:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:09:24.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:09:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:09:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:09:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.594011101000007, 2024-09-16T14:09:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:09:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.325 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:09:24.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:09:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:09:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:09:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:09:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:09:24.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:09:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.331 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.593981385000006, 2024-09-16T14:09:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:09:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:09:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:09:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:09:24.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:09:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:09:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:09:24.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:09:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:09:24.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:09:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:09:24.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:09:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:09:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:09:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:09:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:24.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.593961723000007, 2024-09-16T14:09:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:09:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:09:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:24.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:24.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:24.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:09:24.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:09:24.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:09:24.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:09:24.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:09:24.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:09:24.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:09:24.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:09:24.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:09:24.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:09:24.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 80.585201597999998, 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:24.352 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:09:24.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.500475055999999, 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.357 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:24.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:09:24.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.855471518999998, 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:09:24.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:24.369 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.855384657999998, 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:09:24.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.666078038999999, 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.377 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:09:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.780853817000001, 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:09:24.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:09:24.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:09:24.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.7637906349999994, 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.390 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:09:24.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:09:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:09:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.396 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.396 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:24.398 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:24.398 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:24.391+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:24.499 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:24.495+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:24.506 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:24.506 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:24.506 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:24.506 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:24.516 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:24.520 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-16T14:09:24.536 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.536 INFO:teuthology.orchestra.run.smithi079.stdout:5736 read 0x6ee95 thru 0x7e995 (0xfb01 bytes) 2024-09-16T14:09:24.542 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T14:09:24.543 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-16T14:09:24.543 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:09:24.543 DEBUG:teuthology.orchestra.run.smithi079:> type ceph-objectstore-tool 2024-09-16T14:09:24.547 INFO:teuthology.orchestra.run.smithi079.stdout:5737 read 0xbaeb765 thru 0xbaf71d0 (0xba6c bytes) 2024-09-16T14:09:24.547 INFO:teuthology.orchestra.run.smithi079.stdout:5738 punch from 0x598a496 to 0x59939d6, (0x9540 bytes) 2024-09-16T14:09:24.548 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T14:09:24.548 INFO:teuthology.orchestra.run.smithi079.stdout:5739 write 0x1520681 thru 0x152a450 (0x9dd0 bytes) 2024-09-16T14:09:24.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-16T14:09:24.560 INFO:teuthology.orchestra.run.smithi079.stdout:5740 read 0x6f0ecdf thru 0x6f1bec7 (0xd1e9 bytes) 2024-09-16T14:09:24.574 INFO:teuthology.orchestra.run.smithi079.stdout:5741 writesame 0xee2e668 thru 0xee2f6bf data_size 0x4(0x1058 bytes) 2024-09-16T14:09:24.574 INFO:teuthology.orchestra.run.smithi079.stdout:5742 write 0x1bb04cd thru 0x1bbdb88 (0xd6bc bytes) 2024-09-16T14:09:24.583 INFO:teuthology.orchestra.run.smithi079.stdout:5744 punch from 0xc8c5e48 to 0xc8cbafb, (0x5cb3 bytes) 2024-09-16T14:09:24.583 INFO:teuthology.orchestra.run.smithi079.stdout:5745 writesame 0x7bdd02 thru 0x7c945d data_size 0xa(0xb75c bytes) 2024-09-16T14:09:24.595 INFO:teuthology.orchestra.run.smithi079.stdout:5747 write 0xe609da4 thru 0xe6120b7 (0x8314 bytes) 2024-09-16T14:09:24.600 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:24.597+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:24.602 INFO:teuthology.orchestra.run.smithi079.stdout:5748 read 0x2460377 thru 0x2467fe3 (0x7c6d bytes) 2024-09-16T14:09:24.602 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.602 INFO:teuthology.orchestra.run.smithi079.stdout:5750 write 0x2f520f1 thru 0x2f54f7e (0x2e8e bytes) 2024-09-16T14:09:24.609 INFO:teuthology.orchestra.run.smithi079.stdout:5751 writesame 0x52ac42d thru 0x52b2021 data_size 0x3b(0x5bf5 bytes) 2024-09-16T14:09:24.615 INFO:teuthology.orchestra.run.smithi079.stdout:5753 punch from 0x91d68ce to 0x91d91ac, (0x28de bytes) 2024-09-16T14:09:24.615 INFO:teuthology.orchestra.run.smithi079.stdout:5754 punch from 0x68de5ca to 0x68e935d, (0xad93 bytes) 2024-09-16T14:09:24.615 INFO:teuthology.orchestra.run.smithi079.stdout:5756 read 0x7fd1ba6 thru 0x7fe0e49 (0xf2a4 bytes) 2024-09-16T14:09:24.616 INFO:teuthology.orchestra.run.smithi079.stdout:5757 trunc from 0xee2f6c0 to 0xe529107 2024-09-16T14:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:24.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:24.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:24.649 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.649 INFO:teuthology.orchestra.run.smithi079.stdout:5759 write 0x1e5c121 thru 0x1e66278 (0xa158 bytes) 2024-09-16T14:09:24.656 INFO:teuthology.orchestra.run.smithi079.stdout:5761 read 0xaafab02 thru 0xab062ed (0xb7ec bytes) 2024-09-16T14:09:24.657 INFO:teuthology.orchestra.run.smithi079.stdout:5762 trunc from 0xe529107 to 0xe84956d 2024-09-16T14:09:24.672 INFO:teuthology.orchestra.run.smithi079.stdout:5763 write 0x7db69ee thru 0x7dbaeda (0x44ed bytes) 2024-09-16T14:09:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:5764 write 0x96df478 thru 0x96eef7a (0xfb03 bytes) 2024-09-16T14:09:24.690 INFO:teuthology.orchestra.run.smithi079.stdout:5765 trunc from 0xe84956d to 0xbde3865 2024-09-16T14:09:24.700 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:24.695+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:24.726 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:24.726 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.892653679000006, 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:24.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:24.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:24.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:24.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:24.733 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.892637339999993, 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:24.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:24.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:24.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.892612174999996, 2024-09-16T14:09:24.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:24.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.740 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:24.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:24.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:24.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.891708207999997, 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:24.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:24.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:24.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.891698868999995, 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:24.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.891408747, 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:24.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:24.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:24.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:24.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:24.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.891190053000003, 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:24.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:24.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:24.774 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.884796473999998, 2024-09-16T14:09:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:24.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:24.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:24.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:24.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:24.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:24.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:24.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:24.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:24.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.787 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.880884910999995, 2024-09-16T14:09:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.788 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:24.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:24.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.880853705999996, 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.795 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:24.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:24.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:24.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.801 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.880034265999996, 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:24.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:24.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.879989456999994, 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:24.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:24.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:24.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.876980203000002, 2024-09-16T14:09:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:24.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:24.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.876963243000006, 2024-09-16T14:09:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:24.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:24.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:24.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:24.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:24.829 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.876951747000007, 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:24.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.872204836999998, 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:24.835 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:24.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.864572206000005, 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.841 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:24.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:24.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.864559998999994, 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.846 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:24.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:24.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.862305083999999, 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:24.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:24.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:24.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:24.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:24.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.861 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.861 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:24.861 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.862080832999993, 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:24.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:24.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:24.867 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:24.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:09:24.869 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-16T14:09:24.870 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:24.871 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:24.865+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:24.871 INFO:teuthology.orchestra.run.smithi079.stdout:5766 write 0xa2d23c3 thru 0xa2dafa0 (0x8bde bytes) 2024-09-16T14:09:24.871 INFO:teuthology.orchestra.run.smithi079.stdout:5770 read 0x170ea46 thru 0x1715530 (0x6aeb bytes) 2024-09-16T14:09:24.871 INFO:teuthology.orchestra.run.smithi079.stdout:5771 read 0x33f03f9 thru 0x33f9698 (0x92a0 bytes) 2024-09-16T14:09:24.871 INFO:teuthology.orchestra.run.smithi079.stdout:5773 writesame 0xc380d1a thru 0xc38c468 data_size 0x1(0xb74f bytes) 2024-09-16T14:09:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:5776 read 0x5b0e054 thru 0x5b1da36 (0xf9e3 bytes) 2024-09-16T14:09:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:5777 writesame 0x5a0c3b0 thru 0x5a1b940 data_size 0x7f(0xf591 bytes) 2024-09-16T14:09:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:5778 writesame 0x90e8a02 thru 0x90e9fc7 data_size 0x3(0x15c6 bytes) 2024-09-16T14:09:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:5780 write 0x3bb65eb thru 0x3bbe866 (0x827c bytes) 2024-09-16T14:09:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:5781 write 0x1e51d23 thru 0x1e5f85f (0xdb3d bytes) 2024-09-16T14:09:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:5783 write 0x32a98db thru 0x32b7657 (0xdd7d bytes) 2024-09-16T14:09:24.872 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:24.873 INFO:teuthology.orchestra.run.smithi079.stdout:5785 writesame 0x3ddc4f thru 0x3e5f9b data_size 0x1(0x834d bytes) 2024-09-16T14:09:24.873 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:24.873 INFO:teuthology.orchestra.run.smithi079.stdout:5787 write 0xd72e4bb thru 0xd7337c9 (0x530f bytes) 2024-09-16T14:09:24.874 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:24.874 INFO:teuthology.orchestra.run.smithi079.stdout:5788 read 0x97aeca2 thru 0x97be7c2 (0xfb21 bytes) 2024-09-16T14:09:24.874 INFO:teuthology.orchestra.run.smithi079.stdout:5790 trunc from 0xd7337ca to 0xb2b182d 2024-09-16T14:09:24.874 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:24.875 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:24.878 INFO:teuthology.orchestra.run.smithi079.stdout:5791 write 0xa2ff3f7 thru 0xa3027fd (0x3407 bytes) 2024-09-16T14:09:24.885 INFO:teuthology.orchestra.run.smithi079.stdout:5793 write 0x8cba666 thru 0x8cbf4c5 (0x4e60 bytes) 2024-09-16T14:09:24.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:24.905 INFO:teuthology.orchestra.run.smithi079.stdout:5794 writesame 0xcf06381 thru 0xcf06b66 data_size 0x3(0x7e6 bytes) 2024-09-16T14:09:24.906 INFO:teuthology.orchestra.run.smithi079.stdout:5795 writesame 0x3d1cf6a thru 0x3d28002 data_size 0x1(0xb099 bytes) 2024-09-16T14:09:24.920 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:24.920 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:24.920 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:24.920 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:24.921 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:24.921 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:24.921 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:24.921 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:24.921 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:24.922 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:24.922 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:24.927 INFO:teuthology.orchestra.run.smithi079.stdout:5796 write 0x3325dc2 thru 0x332ce85 (0x70c4 bytes) 2024-09-16T14:09:24.934 INFO:teuthology.orchestra.run.smithi079.stdout:5798 writesame 0x450f3b6 thru 0x451e555 data_size 0x10(0xf1a0 bytes) 2024-09-16T14:09:24.939 INFO:teuthology.orchestra.run.smithi079.stdout:5800 writesame 0xea8c6b thru 0xeb7e26 data_size 0xa2(0xf1bc bytes) 2024-09-16T14:09:24.947 INFO:teuthology.orchestra.run.smithi079.stdout:5802 read 0x61ec0d7 thru 0x61f995a (0xd884 bytes) 2024-09-16T14:09:24.947 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.947 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.947 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.947 INFO:teuthology.orchestra.run.smithi079.stdout:5808 read 0x724cb97 thru 0x725c18f (0xf5f9 bytes) 2024-09-16T14:09:24.947 INFO:teuthology.orchestra.run.smithi079.stdout:5809 read 0x4292a15 thru 0x4296ab4 (0x40a0 bytes) 2024-09-16T14:09:24.948 INFO:teuthology.orchestra.run.smithi079.stdout:5810 read 0x3ea933f thru 0x3eb759d (0xe25f bytes) 2024-09-16T14:09:24.948 INFO:teuthology.orchestra.run.smithi079.stdout:5811 read 0xc5fb7f4 thru 0xc5fe559 (0x2d66 bytes) 2024-09-16T14:09:24.948 INFO:teuthology.orchestra.run.smithi079.stdout:5812 punch from 0x1a371e9 to 0x1a41465, (0xa27c bytes) 2024-09-16T14:09:24.948 INFO:teuthology.orchestra.run.smithi079.stdout:5813 read 0x8e36a58 thru 0x8e468e9 (0xfe92 bytes) 2024-09-16T14:09:24.948 INFO:teuthology.orchestra.run.smithi079.stdout:5818 write 0xaf0397c thru 0xaf064b5 (0x2b3a bytes) 2024-09-16T14:09:24.951 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.951 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:24.951 INFO:teuthology.orchestra.run.smithi079.stdout:5823 read 0x1c8ef2e thru 0x1c986de (0x97b1 bytes) 2024-09-16T14:09:24.970 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:24.965+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:24.973 INFO:teuthology.orchestra.run.smithi079.stdout:5824 trunc from 0xcf06b67 to 0x72c244 2024-09-16T14:09:24.981 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:24.981 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:24.993 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:25.040 INFO:teuthology.orchestra.run.smithi079.stdout:5825 write 0x6b21f55 thru 0x6b26717 (0x47c3 bytes) 2024-09-16T14:09:25.044 INFO:teuthology.orchestra.run.smithi079.stdout:5826 read 0x3b1508b thru 0x3b1e389 (0x92ff bytes) 2024-09-16T14:09:25.044 INFO:teuthology.orchestra.run.smithi079.stdout:5827 read 0x5222200 thru 0x522a863 (0x8664 bytes) 2024-09-16T14:09:25.044 INFO:teuthology.orchestra.run.smithi079.stdout:5829 read 0x1580060 thru 0x158a73b (0xa6dc bytes) 2024-09-16T14:09:25.064 INFO:teuthology.orchestra.run.smithi079.stdout:5831 write 0xed3d4f8 thru 0xed4897e (0xb487 bytes) 2024-09-16T14:09:25.070 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:25.067+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:25.073 INFO:teuthology.orchestra.run.smithi079.stdout:5832 read 0x43a2dcd thru 0x43aa356 (0x758a bytes) 2024-09-16T14:09:25.073 INFO:teuthology.orchestra.run.smithi079.stdout:5833 read 0x7b14576 thru 0x7b1fcfe (0xb789 bytes) 2024-09-16T14:09:25.073 INFO:teuthology.orchestra.run.smithi079.stdout:5834 trunc from 0xed4897f to 0xebe2bad 2024-09-16T14:09:25.092 INFO:teuthology.orchestra.run.smithi079.stdout:5835 read 0xa326efe thru 0xa328bd3 (0x1cd6 bytes) 2024-09-16T14:09:25.092 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.092 INFO:teuthology.orchestra.run.smithi079.stdout:5837 writesame 0x84262fb thru 0x842f409 data_size 0x7(0x910f bytes) 2024-09-16T14:09:25.095 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:25.095 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:25.100 INFO:teuthology.orchestra.run.smithi079.stdout:5838 read 0xb8a2ef3 thru 0xb8a66e1 (0x37ef bytes) 2024-09-16T14:09:25.100 INFO:teuthology.orchestra.run.smithi079.stdout:5839 write 0x2242b3e thru 0x224c046 (0x9509 bytes) 2024-09-16T14:09:25.105 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:25.128 INFO:teuthology.orchestra.run.smithi079.stdout:5840 trunc from 0xebe2bad to 0x20b05d1 2024-09-16T14:09:25.161 INFO:teuthology.orchestra.run.smithi079.stdout:5841 writesame 0x73358d3 thru 0x733f24f data_size 0x1(0x997d bytes) 2024-09-16T14:09:25.163 INFO:teuthology.orchestra.run.smithi079.stdout:5842 trunc from 0x733f250 to 0x62a9608 2024-09-16T14:09:25.171 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:25.169+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:25.172 INFO:teuthology.orchestra.run.smithi079.stdout:5843 writesame 0x2c66bac thru 0x2c6e95c data_size 0x1(0x7db1 bytes) 2024-09-16T14:09:25.197 INFO:teuthology.orchestra.run.smithi079.stdout:5844 writesame 0x4b3096 thru 0x4b5eca data_size 0x3(0x2e35 bytes) 2024-09-16T14:09:25.206 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:25.217 INFO:teuthology.orchestra.run.smithi079.stdout:5846 trunc from 0x62a9608 to 0xd0ef06e 2024-09-16T14:09:25.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:25.227 INFO:teuthology.orchestra.run.smithi079.stdout:5848 writesame 0x4a5cb3c thru 0x4a60fed data_size 0x9(0x44b2 bytes) 2024-09-16T14:09:25.234 INFO:teuthology.orchestra.run.smithi079.stdout:5849 read 0x54c0eb thru 0x558f50 (0xce66 bytes) 2024-09-16T14:09:25.235 INFO:teuthology.orchestra.run.smithi079.stdout:5851 write 0x70de762 thru 0x70eae34 (0xc6d3 bytes) 2024-09-16T14:09:25.242 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.243 INFO:teuthology.orchestra.run.smithi079.stdout:5855 write 0x31a32c0 thru 0x31b167f (0xe3c0 bytes) 2024-09-16T14:09:25.248 INFO:teuthology.orchestra.run.smithi079.stdout:5858 write 0x90b1a82 thru 0x90b33fd (0x197c bytes) 2024-09-16T14:09:25.263 INFO:teuthology.orchestra.run.smithi079.stdout:5859 write 0xd11d639 thru 0xd122313 (0x4cdb bytes) 2024-09-16T14:09:25.272 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:25.269+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:25.288 INFO:teuthology.orchestra.run.smithi079.stdout:5860 trunc from 0xd122314 to 0x3322a56 2024-09-16T14:09:25.310 INFO:teuthology.orchestra.run.smithi079.stdout:5861 read 0x1781c thru 0x25ba8 (0xe38d bytes) 2024-09-16T14:09:25.311 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.312 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.317 INFO:teuthology.orchestra.run.smithi079.stdout:5865 trunc from 0x3322a56 to 0x5c787f3 2024-09-16T14:09:25.326 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:25.326 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:25.333 INFO:teuthology.orchestra.run.smithi079.stdout:5867 read 0x2a5386f thru 0x2a60c30 (0xd3c2 bytes) 2024-09-16T14:09:25.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:25.349 INFO:teuthology.orchestra.run.smithi079.stdout:5869 write 0x99983af thru 0x99a0b43 (0x8795 bytes) 2024-09-16T14:09:25.358 INFO:teuthology.orchestra.run.smithi079.stdout:5870 punch from 0x62ae7da to 0x62affb6, (0x17dc bytes) 2024-09-16T14:09:25.358 INFO:teuthology.orchestra.run.smithi079.stdout:5871 write 0x613d035 thru 0x613f8e9 (0x28b5 bytes) 2024-09-16T14:09:25.364 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.372 INFO:teuthology.orchestra.run.smithi079.stdout:5875 trunc from 0x99a0b44 to 0xe788709 2024-09-16T14:09:25.378 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:25.377+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:25.383 INFO:teuthology.orchestra.run.smithi079.stdout:5876 write 0x375aea9 thru 0x3768cfb (0xde53 bytes) 2024-09-16T14:09:25.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:09:25.379+0000 7fb14b004300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T14:09:25.392 INFO:teuthology.orchestra.run.smithi079.stdout:5877 write 0x3631be5 thru 0x3631efd (0x319 bytes) 2024-09-16T14:09:25.398 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.398 INFO:teuthology.orchestra.run.smithi079.stdout:5879 read 0xc700f0 thru 0xc7d186 (0xd097 bytes) 2024-09-16T14:09:25.398 INFO:teuthology.orchestra.run.smithi079.stdout:5881 punch from 0x5032a87 to 0x5038786, (0x5cff bytes) 2024-09-16T14:09:25.398 INFO:teuthology.orchestra.run.smithi079.stdout:5882 write 0xa403e65 thru 0xa410fe4 (0xd180 bytes) 2024-09-16T14:09:25.404 INFO:teuthology.orchestra.run.smithi079.stdout:5883 read 0xac65320 thru 0xac6c67e (0x735f bytes) 2024-09-16T14:09:25.405 INFO:teuthology.orchestra.run.smithi079.stdout:5884 writesame 0xd249da6 thru 0xd24ed94 data_size 0x13(0x4fef bytes) 2024-09-16T14:09:25.409 INFO:teuthology.orchestra.run.smithi079.stdout:5885 punch from 0x761d1c3 to 0x76222f8, (0x5135 bytes) 2024-09-16T14:09:25.409 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.409 INFO:teuthology.orchestra.run.smithi079.stdout:5888 punch from 0x471d7ba to 0x4727a3f, (0xa285 bytes) 2024-09-16T14:09:25.417 INFO:teuthology.orchestra.run.smithi079.stdout:5889 trunc from 0xe788709 to 0xaadaf56 2024-09-16T14:09:25.437 INFO:teuthology.orchestra.run.smithi079.stdout:5890 write 0x23fa1d0 thru 0x240345d (0x928e bytes) 2024-09-16T14:09:25.450 INFO:teuthology.orchestra.run.smithi079.stdout:5891 write 0x525a080 thru 0x5267641 (0xd5c2 bytes) 2024-09-16T14:09:25.473 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:25.467+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:25.481 INFO:teuthology.orchestra.run.smithi079.stdout:5892 writesame 0xde31512 thru 0xde3fdc3 data_size 0x73(0xe8b2 bytes) 2024-09-16T14:09:25.481 INFO:teuthology.orchestra.run.smithi079.stdout:5893 writesame 0x825ad50 thru 0x825ba65 data_size 0x5(0xd16 bytes) 2024-09-16T14:09:25.487 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:25.487 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:25.488 INFO:teuthology.orchestra.run.smithi079.stdout:5894 read 0xd689e38 thru 0xd68d9a8 (0x3b71 bytes) 2024-09-16T14:09:25.489 INFO:teuthology.orchestra.run.smithi079.stdout:5895 writesame 0x71ace8c thru 0x71bc334 data_size 0x1(0xf4a9 bytes) 2024-09-16T14:09:25.501 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:25.534 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.553 INFO:teuthology.orchestra.run.smithi079.stdout:5898 trunc from 0xde3fdc4 to 0x5d93d32 2024-09-16T14:09:25.574 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:25.573+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:25.597 INFO:teuthology.orchestra.run.smithi079.stdout:5899 writesame 0x72a465b thru 0x72a64e7 data_size 0xb(0x1e8d bytes) 2024-09-16T14:09:25.597 INFO:teuthology.orchestra.run.smithi079.stdout:5900 read 0x194c096 thru 0x1956f5d (0xaec8 bytes) 2024-09-16T14:09:25.597 INFO:teuthology.orchestra.run.smithi079.stdout:5902 write 0x5eeeea0 thru 0x5ef10e5 (0x2246 bytes) 2024-09-16T14:09:25.601 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:25.601 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:25.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:25.630 INFO:teuthology.orchestra.run.smithi079.stdout:5907 writesame 0xe8441a5 thru 0xe84c1d0 data_size 0x1a(0x802c bytes) 2024-09-16T14:09:25.630 INFO:teuthology.orchestra.run.smithi079.stdout:5914 punch from 0x372c437 to 0x372fed6, (0x3a9f bytes) 2024-09-16T14:09:25.630 INFO:teuthology.orchestra.run.smithi079.stdout:5915 read 0xa7d21c7 thru 0xa7dc7ee (0xa628 bytes) 2024-09-16T14:09:25.630 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.647 INFO:teuthology.orchestra.run.smithi079.stdout:5917 trunc from 0xe84c1d1 to 0x6fceef3 2024-09-16T14:09:25.677 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:25.673+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:25.679 INFO:teuthology.orchestra.run.smithi079.stdout:5918 writesame 0xbe3aca4 thru 0xbe4796d data_size 0xb(0xccca bytes) 2024-09-16T14:09:25.679 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.679 INFO:teuthology.orchestra.run.smithi079.stdout:5920 read 0xac76da7 thru 0xac8130a (0xa564 bytes) 2024-09-16T14:09:25.680 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.695 INFO:teuthology.orchestra.run.smithi079.stdout:5922 write 0xe9626e3 thru 0xe9655d0 (0x2eee bytes) 2024-09-16T14:09:25.701 INFO:teuthology.orchestra.run.smithi079.stdout:5923 write 0xb12171a thru 0xb127aab (0x6392 bytes) 2024-09-16T14:09:25.715 INFO:teuthology.orchestra.run.smithi079.stdout:5924 trunc from 0xe9655d1 to 0xa335211 2024-09-16T14:09:25.734 INFO:teuthology.orchestra.run.smithi079.stdout:5925 punch from 0x443839a to 0x4443dc2, (0xba28 bytes) 2024-09-16T14:09:25.736 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:25.737 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:25.745 INFO:teuthology.orchestra.run.smithi079.stdout:5926 trunc from 0xa335211 to 0x3c83dce 2024-09-16T14:09:25.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:25.761 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:09:25.769 INFO:teuthology.orchestra.run.smithi079.stdout:5927 trunc from 0x3c83dce to 0x3afa413 2024-09-16T14:09:25.778 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:25.773+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:25.787 INFO:teuthology.orchestra.run.smithi079.stdout:5928 read 0x2b8a279 thru 0x2b8c88d (0x2615 bytes) 2024-09-16T14:09:25.788 INFO:teuthology.orchestra.run.smithi079.stdout:5929 read 0x3114401 thru 0x31163ef (0x1fef bytes) 2024-09-16T14:09:25.788 INFO:teuthology.orchestra.run.smithi079.stdout:5930 punch from 0x887609 to 0x894da4, (0xd79b bytes) 2024-09-16T14:09:25.788 INFO:teuthology.orchestra.run.smithi079.stdout:5931 writesame 0x27f877 thru 0x28aad5 data_size 0x1f(0xb25f bytes) 2024-09-16T14:09:25.791 INFO:teuthology.orchestra.run.smithi079.stdout:5933 punch from 0x3ab9a27 to 0x3abe7a3, (0x4d7c bytes) 2024-09-16T14:09:25.791 INFO:teuthology.orchestra.run.smithi079.stdout:5934 punch from 0x138d226 to 0x13916c3, (0x449d bytes) 2024-09-16T14:09:25.792 INFO:teuthology.orchestra.run.smithi079.stdout:5936 writesame 0x204a997 thru 0x2051f1c data_size 0x31(0x7586 bytes) 2024-09-16T14:09:25.797 INFO:teuthology.orchestra.run.smithi079.stdout:5937 writesame 0x16aaf1a thru 0x16b397e data_size 0x1(0x8a65 bytes) 2024-09-16T14:09:25.843 INFO:teuthology.orchestra.run.smithi079.stdout:5938 write 0x6f091a3 thru 0x6f18651 (0xf4af bytes) 2024-09-16T14:09:25.849 INFO:teuthology.orchestra.run.smithi079.stdout:5939 read 0x3d0b118 thru 0x3d17282 (0xc16b bytes) 2024-09-16T14:09:25.849 INFO:teuthology.orchestra.run.smithi079.stdout:5940 writesame 0xe2e0e2 thru 0xe35b09 data_size 0xc(0x7a28 bytes) 2024-09-16T14:09:25.856 INFO:teuthology.orchestra.run.smithi079.stdout:5941 write 0x5ffc572 thru 0x5ffcd19 (0x7a8 bytes) 2024-09-16T14:09:25.867 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:25.867 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:25.867 INFO:teuthology.orchestra.run.smithi079.stdout:5942 trunc from 0x6f18652 to 0x9a53255 2024-09-16T14:09:25.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:25.880 INFO:teuthology.orchestra.run.smithi079.stdout:5943 read 0x48be040 thru 0x48c8b57 (0xab18 bytes) 2024-09-16T14:09:25.881 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:25.877+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:25.884 INFO:teuthology.orchestra.run.smithi079.stdout:5945 trunc from 0x9a53255 to 0x858f8e8 2024-09-16T14:09:25.901 INFO:teuthology.orchestra.run.smithi079.stdout:5947 trunc from 0x858f8e8 to 0x4a9c6da 2024-09-16T14:09:25.920 INFO:teuthology.orchestra.run.smithi079.stdout:5951 writesame 0x418ebe4 thru 0x41949a2 data_size 0x1(0x5dbf bytes) 2024-09-16T14:09:25.944 INFO:teuthology.orchestra.run.smithi079.stdout:5953 write 0x452e556 thru 0x453b5d9 (0xd084 bytes) 2024-09-16T14:09:25.951 INFO:teuthology.orchestra.run.smithi079.stdout:5954 punch from 0x3392489 to 0x3396997, (0x450e bytes) 2024-09-16T14:09:25.971 INFO:teuthology.orchestra.run.smithi079.stdout:5956 trunc from 0x4a9c6da to 0xea090f5 2024-09-16T14:09:25.980 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:25.980 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:25.981 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:25.975+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:25.981 INFO:teuthology.orchestra.run.smithi079.stdout:5958 write 0x7b94176 thru 0x7b9ad97 (0x6c22 bytes) 2024-09-16T14:09:25.986 INFO:teuthology.orchestra.run.smithi079.stdout:5959 read 0x55166fd thru 0x5520bc5 (0xa4c9 bytes) 2024-09-16T14:09:25.986 INFO:teuthology.orchestra.run.smithi079.stdout:5960 read 0xe43b978 thru 0xe43da53 (0x20dc bytes) 2024-09-16T14:09:25.986 INFO:teuthology.orchestra.run.smithi079.stdout:5961 read 0x7bc9a57 thru 0x7bd37a4 (0x9d4e bytes) 2024-09-16T14:09:25.987 INFO:teuthology.orchestra.run.smithi079.stdout:5962 write 0xe431577 thru 0xe44004b (0xead5 bytes) 2024-09-16T14:09:25.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:25.993 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:25.993 INFO:teuthology.orchestra.run.smithi079.stdout:5964 punch from 0xc2818e5 to 0xc28b2a5, (0x99c0 bytes) 2024-09-16T14:09:25.993 INFO:teuthology.orchestra.run.smithi079.stdout:5965 read 0x6bdf790 thru 0x6bed7fa (0xe06b bytes) 2024-09-16T14:09:25.993 INFO:teuthology.orchestra.run.smithi079.stdout:5966 read 0x60bde8f thru 0x60c94f7 (0xb669 bytes) 2024-09-16T14:09:25.994 INFO:teuthology.orchestra.run.smithi079.stdout:5968 read 0x4f05c4c thru 0x4f07b9b (0x1f50 bytes) 2024-09-16T14:09:25.994 INFO:teuthology.orchestra.run.smithi079.stdout:5970 write 0xd9c43fc thru 0xd9d10d7 (0xccdc bytes) 2024-09-16T14:09:26.000 INFO:teuthology.orchestra.run.smithi079.stdout:5971 write 0x7b8ec8d thru 0x7b9a117 (0xb48b bytes) 2024-09-16T14:09:26.002 INFO:teuthology.orchestra.run.smithi079.stdout:5972 writesame 0x62e1b40 thru 0x62f0e49 data_size 0xd(0xf30a bytes) 2024-09-16T14:09:26.010 INFO:teuthology.orchestra.run.smithi079.stdout:5973 writesame 0x89cbb07 thru 0x89d614d data_size 0x15(0xa647 bytes) 2024-09-16T14:09:26.017 INFO:teuthology.orchestra.run.smithi079.stdout:5976 read 0x241bf49 thru 0x242481d (0x88d5 bytes) 2024-09-16T14:09:26.017 INFO:teuthology.orchestra.run.smithi079.stdout:5977 write 0x5491b5e thru 0x54a0fcc (0xf46f bytes) 2024-09-16T14:09:26.024 INFO:teuthology.orchestra.run.smithi079.stdout:5979 read 0x4a26dd7 thru 0x4a33374 (0xc59e bytes) 2024-09-16T14:09:26.046 INFO:teuthology.orchestra.run.smithi079.stdout:5980 trunc from 0xea090f5 to 0x729e0b 2024-09-16T14:09:26.079 INFO:teuthology.orchestra.run.smithi079.stdout:5981 read 0x129491 thru 0x12d7b2 (0x4322 bytes) 2024-09-16T14:09:26.081 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:26.077+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:26.093 INFO:teuthology.orchestra.run.smithi079.stdout:5982 trunc from 0x729e0b to 0x8b516f5 2024-09-16T14:09:26.098 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:26.098 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:26.103 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:26.103 INFO:teuthology.orchestra.run.smithi079.stdout:5984 punch from 0x44a41d4 to 0x44b3b4f, (0xf97b bytes) 2024-09-16T14:09:26.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:26.114 INFO:teuthology.orchestra.run.smithi079.stdout:5988 writesame 0x8d2701a thru 0x8d2b40e data_size 0x3(0x43f5 bytes) 2024-09-16T14:09:26.114 INFO:teuthology.orchestra.run.smithi079.stdout:5989 write 0x55a6995 thru 0x55b32da (0xc946 bytes) 2024-09-16T14:09:26.134 INFO:teuthology.orchestra.run.smithi079.stdout:5990 trunc from 0x8d2b40f to 0x143f5b4 2024-09-16T14:09:26.183 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:26.181+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:26.190 INFO:teuthology.orchestra.run.smithi079.stdout:5991 write 0xe825d8c thru 0xe8332b7 (0xd52c bytes) 2024-09-16T14:09:26.197 INFO:teuthology.orchestra.run.smithi079.stdout:5992 read 0x328d2e3 thru 0x329765f (0xa37d bytes) 2024-09-16T14:09:26.197 INFO:teuthology.orchestra.run.smithi079.stdout:5993 write 0x491d936 thru 0x4920b90 (0x325b bytes) 2024-09-16T14:09:26.208 INFO:teuthology.orchestra.run.smithi079.stdout:5994 write 0x745d81c thru 0x7460977 (0x315c bytes) 2024-09-16T14:09:26.215 INFO:teuthology.orchestra.run.smithi079.stdout:5995 write 0x85fc7e0 thru 0x85fdd5e (0x157f bytes) 2024-09-16T14:09:26.229 INFO:teuthology.orchestra.run.smithi079.stdout:5996 trunc from 0xe8332b8 to 0x7d20019 2024-09-16T14:09:26.235 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:26.235 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:26.246 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:26.248 INFO:teuthology.orchestra.run.smithi079.stdout:5997 read 0x341abcb thru 0x342a324 (0xf75a bytes) 2024-09-16T14:09:26.248 INFO:teuthology.orchestra.run.smithi079.stdout:compare and write disabled 2024-09-16T14:09:26.248 INFO:teuthology.orchestra.run.smithi079.stdout:5999 read 0x2c99744 thru 0x2c9c876 (0x3133 bytes) 2024-09-16T14:09:26.264 INFO:teuthology.orchestra.run.smithi079.stdout:6000 writesame 0xb276304 thru 0xb283098 data_size 0x3(0xcd95 bytes) 2024-09-16T14:09:26.283 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:26.281+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:26.337 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:26.337 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:26.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:26.385 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:26.381+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:26.389 INFO:teuthology.orchestra.run.smithi079.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-09-16T14:09:26.485 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:26.479+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:26.527 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:26.528 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:26.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:26.586 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:26.583+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:26.631 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:26.631 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:26.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:26.687 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:26.683+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:26.741 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:26.741 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:26.750 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:26.795 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:26.793+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:26.866 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:26.866 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:26.876 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:26.889 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:26.883+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:26.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:26.973 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:26.973 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:26.983 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:26.989 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:26.985+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:27.070 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:27.084 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:27.084 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:27.085 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:27.091 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:27.089+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:27.093 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:27.104 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:27.123 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:27.144 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:27.165 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:27.183 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:27.183 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:27.183 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:27.183 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:27.183 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:27.184 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:27.184 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:27.184 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:27.184 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:27.187 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:27.187 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:27.191 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:27.187+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:27.238 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:27.238 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:27.238 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:27.247 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:27.293 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:27.289+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:27.344 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:27.344 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:27.344 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:27.344 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:27.344 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:27.353 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:27.398 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:27.397+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:27.456 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:27.457 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:27.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:27.457 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:27.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.457 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.457 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:27.457 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.623403159999995, 2024-09-16T14:09:27.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:27.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:27.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:27.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.623386820999997, 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:27.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:27.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.623361656, 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.469 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:27.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.622457689000001, 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:27.474 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:27.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:27.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.622448349999999, 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:27.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.622158228000004, 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:27.485 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:27.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:27.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.621939534000006, 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:27.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.615545955000002, 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:27.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:27.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:27.501 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.611634391999999, 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:27.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.507 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.611603187, 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:27.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.610783746999999, 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.610738937999997, 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:27.518 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:27.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:27.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:27.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:27.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.607729684000006, 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.528 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:27.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.607712723999995, 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:27.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:27.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:27.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:27.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.607701227999996, 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:27.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:27.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:27.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.602954318000002, 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:27.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:27.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:27.549 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.595321686999995, 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:27.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:27.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:27.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.595309479999997, 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:27.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:27.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:27.560 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.593054565000003, 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:27.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:27.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.592830313999997, 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:27.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:27.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:27.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:27.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.571 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.571 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.571 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:27.572 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:27.573 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:27.567+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:27.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:09:27.571+0000 7fb14b004300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T14:09:27.673 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:27.667+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:27.678 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:27.679 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:27.688 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:27.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:09:27.777 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:27.773+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:27.788 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:27.789 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:27.789 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:27.789 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:27.789 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:27.797 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:27.875 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:27.873+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:27.952 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.574217285000003, 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:27.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:09:27.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.570285866999996, 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:09:27.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:09:27.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.570261674999998, 2024-09-16T14:09:27.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:09:27.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.566542404000003, 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:09:27.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.562625639000004, 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.972 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:09:27.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:09:27.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:09:27.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:09:27.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:09:27.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:09:27.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.558243360000006, 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:09:27.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:27.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:09:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:09:27.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:09:27.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:09:27.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:09:27.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.558219519000005, 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:09:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:09:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:09:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:09:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.558204330999999, 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:09:27.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.558189744999993, 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:09:27.995 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:27.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:09:27.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:09:27.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.000 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.558086094999993, 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:09:28.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:09:28.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.005 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.558058368999994, 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:09:28.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:09:28.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:28.011 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.555358038999998, 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:09:28.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:09:28.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:09:28.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:09:28.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:28.017 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.555317872000003, 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:09:28.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.555228157000002, 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.023 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:09:28.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:28.028 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.554719168999995, 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:09:28.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.033 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.034 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:28.035 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:28.035 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.546621798999993, 2024-09-16T14:09:28.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:09:28.035 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.035 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:28.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:09:28.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:09:28.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 79.544453507, 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:09:28.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:09:28.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.047 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.394892045999999, 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.048 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:09:28.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:09:28.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [watch reconnect cookie 140383155395456 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2648828180000002, 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.013377462, 2024-09-16T14:09:28.055 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:09:28.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:09:28.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.694814+0000", 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003026674 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.695138+0000", 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032383800000000001 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697164+0000", 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020260149999999999 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2590196840000001, 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.007999884, 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:09:28.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:28.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:09:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696135+0000", 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003762636 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696215+0000", 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0119999999999999e-05 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697528+0000", 2024-09-16T14:09:28.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001312887 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697582+0000", 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4409e-05 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697649+0000", 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7026000000000005e-05 2024-09-16T14:09:28.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.070 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.070 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.070 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.070 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.070 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:28.071 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:28.072 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:28.067+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:28.173 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:28.169+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:28.181 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:28.181 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:28.181 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:28.182 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:28.190 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:28.274 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:28.269+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:28.291 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:28.291 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:28.291 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:28.291 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:28.291 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:28.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:28.374 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:28.371+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:28.423 INFO:teuthology.orchestra.run.smithi079.stdout:3.f 2024-09-16T14:09:28.423 INFO:teuthology.orchestra.run.smithi079.stdout:3.e 2024-09-16T14:09:28.423 INFO:teuthology.orchestra.run.smithi079.stdout:3.d 2024-09-16T14:09:28.423 INFO:teuthology.orchestra.run.smithi079.stdout:3.c 2024-09-16T14:09:28.423 INFO:teuthology.orchestra.run.smithi079.stdout:3.3 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:3.1d 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:3.b 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:3.1c 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:3.13 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:3.15 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:2.7 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:3.1b 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:3.14 2024-09-16T14:09:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:2.6 2024-09-16T14:09:28.450 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:28.450 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:28.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:28.450 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:28.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.450 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:28.450 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:28.451 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 127.034993542, 2024-09-16T14:09:28.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:28.451 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.451 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:28.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.452 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:28.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.452 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:28.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:28.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:28.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:28.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:28.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:28.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:28.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:28.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:28.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:28.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:28.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:28.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:28.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:28.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:28.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:28.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:28.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:28.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:28.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:28.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:28.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:28.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:28.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:28.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:28.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:28.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:28.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:28.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:28.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:28.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:28.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:28.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:28.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.462 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.197031218999996, 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.463 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:28.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:28.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:28.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:28.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:28.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:28.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:28.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:28.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:28.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:28.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:28.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:28.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:28.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:28.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:28.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.669792231000002, 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:28.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.475 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.669769000000002, 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:28.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:28.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:28.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:28.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.669746052999997, 2024-09-16T14:09:28.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:28.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:28.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:28.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:28.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:28.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:28.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:28.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:28.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:28.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.487 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.488 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.488 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:28.488 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.669729544000006, 2024-09-16T14:09:28.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:28.488 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.488 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.488 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.489 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:28.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.489 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:28.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:28.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:28.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:28.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:28.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:28.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:28.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.494 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.669711918000004, 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:28.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:28.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.663796173999998, 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:28.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.663778769000004, 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:28.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.663424402999993, 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.660280544000003, 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.512 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:28.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.660265257999995, 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.660248848999998, 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:28.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:28.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.660173651999997, 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:28.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.660076105000002, 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:28.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.534 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.660050863999999, 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:28.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:28.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.660021810000003, 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.539 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.656757820999999, 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:28.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:28.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.656739619999996, 2024-09-16T14:09:28.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:28.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.552 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.656700442000002, 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:28.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:28.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:28.557 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:28.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:28.559 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:28.557+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:28.659 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:28.657+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:28.664 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:28.664 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:28.664 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:28.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:28.760 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:28.755+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:28.771 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:28.771 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:28.771 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:28.771 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:28.771 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:28.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:28.787 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:09:28.861 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:28.855+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:28.875 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.635705762999997, 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:28.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:28.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.27555272, 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275543720999998, 2024-09-16T14:09:28.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:28.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.27553537, 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:28.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:28.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275502686000003, 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:28.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275493449000002, 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:28.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.903 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275484917, 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:28.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.908 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275476339999997, 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:28.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275466979000001, 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:28.913 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:28.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275458866999998, 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:28.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:28.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:28.922 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275447163999999, 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:28.926 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275438551000001, 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.927 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:28.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:28.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275430444999998, 2024-09-16T14:09:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275422102999997, 2024-09-16T14:09:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275413565000001, 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275405233000001, 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275396991000001, 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275387428000002, 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275378046999997, 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.275368405999998, 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:28.969 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:28.971 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:28.967+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:28.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:28.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.3 --file /var/log/ceph/exp.3.3.0 2024-09-16T14:09:29.072 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:29.069+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:29.079 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:29.079 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:29.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:29.173 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:29.169+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:29.187 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:29.187 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:29.187 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:29.187 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:29.187 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:29.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:29.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:29.274 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:29.271+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:29.323 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:29.339 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:29.340 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:29.346 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:29.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:29.372 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:29.374 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:29.373+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:29.407 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:29.440 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:29.443 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:29.443 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:29.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:29.472 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:29.506 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:29.506 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:29.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:29.507 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:29.503+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:29.510 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:29.510 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:29.575 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:29.575 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:29.576 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:29.571+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:29.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:29.677 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:29.671+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:29.678 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:29.678 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:29.687 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:29.778 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:29.773+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:29.799 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:29.799 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:29.810 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:29.879 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:29.877+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:29.909 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:29.909 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:29.919 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:29.982 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:29.979+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:30.064 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:30.064 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:30.074 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:30.083 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:30.079+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:30.172 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:30.173 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:30.176 INFO:teuthology.orchestra.run.smithi079.stderr:Exporting 3.3 info 3.3( v 121'978 (0'0,121'978] local-lis/les=23/24 n=104 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-16T14:09:30.181 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:30.183 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:30.181+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:30.199 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0101559:::rbd_data.11361fa8f1cb.000000000000001f:head# 2024-09-16T14:09:30.199 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0140dcc:::rbd_data.11364e09a271.0000000000000002:30# 2024-09-16T14:09:30.200 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0140dcc:::rbd_data.11364e09a271.0000000000000002:head# 2024-09-16T14:09:30.200 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0231641:::rbd_data.1136f033099c.0000000000000032:head# 2024-09-16T14:09:30.200 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c04552a4:::rbd_object_map.1136dd12cea0:head# 2024-09-16T14:09:30.200 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c04959d7:::rbd_data.113630ceda18.0000000000000073:25# 2024-09-16T14:09:30.201 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c04959d7:::rbd_data.113630ceda18.0000000000000073:head# 2024-09-16T14:09:30.201 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0783ec2:::rbd_data.11361fa8f1cb.0000000000000076:head# 2024-09-16T14:09:30.201 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0977406:::rbd_data.1136494ba971.000000000000000f:7# 2024-09-16T14:09:30.202 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0977406:::rbd_data.1136494ba971.000000000000000f:head# 2024-09-16T14:09:30.202 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0c9ab1e:::rbd_data.113677f46bcd.0000000000000008:27# 2024-09-16T14:09:30.211 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0c9ab1e:::rbd_data.113677f46bcd.0000000000000008:head# 2024-09-16T14:09:30.211 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0d084e0:::rbd_data.11361fa8f1cb.000000000000002b:head# 2024-09-16T14:09:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0d782ac:::rbd_data.1136a570af09.0000000000000001:head# 2024-09-16T14:09:30.213 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0e631c4:::rbd_header.1136494ba971:head# 2024-09-16T14:09:30.213 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0ea02dd:::rbd_data.113630ceda18.0000000000000002:head# 2024-09-16T14:09:30.215 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c0f16a19:::rbd_data.1136f033099c.0000000000000000:head# 2024-09-16T14:09:30.216 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c10e1f6a:::rbd_data.11362e0d5ebf.0000000000000075:head# 2024-09-16T14:09:30.216 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c12cb214:::rbd_data.11363617f81b.00000000000000dc:head# 2024-09-16T14:09:30.216 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1527b88:::rbd_data.11367eb52311.0000000000000134:head# 2024-09-16T14:09:30.216 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c16144c7:::rbd_data.11369618fede.000000000000000d:head# 2024-09-16T14:09:30.217 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1b850a0:::rbd_data.11362defb50e.000000000000009c:head# 2024-09-16T14:09:30.218 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1c67fb6:::rbd_data.11362e0d5ebf.0000000000000006:head# 2024-09-16T14:09:30.218 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1c80357:::rbd_data.113612ae6a73.0000000000000016:13# 2024-09-16T14:09:30.219 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1c80357:::rbd_data.113612ae6a73.0000000000000016:head# 2024-09-16T14:09:30.219 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1d2593c:::rbd_data.11368cf3b67c.0000000000000008:1b# 2024-09-16T14:09:30.222 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1d2593c:::rbd_data.11368cf3b67c.0000000000000008:head# 2024-09-16T14:09:30.224 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1d2c68b:::rbd_data.11361fa8f1cb.00000000000000f4:head# 2024-09-16T14:09:30.224 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1ecc080:::rbd_data.113619b3ee73.0000000000000041:16# 2024-09-16T14:09:30.224 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1ecc080:::rbd_data.113619b3ee73.0000000000000041:head# 2024-09-16T14:09:30.224 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c1eec317:::rbd_data.11369618fede.0000000000000014:head# 2024-09-16T14:09:30.224 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c203ff01:::rbd_data.11361fb616f9.000000000000000a:14# 2024-09-16T14:09:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c203ff01:::rbd_data.11361fb616f9.000000000000000a:head# 2024-09-16T14:09:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2172984:::rbd_object_map.1136a570af09:head# 2024-09-16T14:09:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c259c83c:::rbd_data.113699bfa25d.0000000000000014:head# 2024-09-16T14:09:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c26647c7:::rbd_data.11361fa8f1cb.0000000000000178:head# 2024-09-16T14:09:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c267fc67:::rbd_data.113619b3ee73.000000000000003d:16# 2024-09-16T14:09:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c267fc67:::rbd_data.113619b3ee73.000000000000003d:head# 2024-09-16T14:09:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c26990c5:::rbd_data.11361fa8f1cb.00000000000000e7:head# 2024-09-16T14:09:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2b8aec4:::rbd_header.1136a719415e:head# 2024-09-16T14:09:30.231 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2c37deb:::rbd_data.11369618fede.0000000000000015:head# 2024-09-16T14:09:30.231 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2c5f3b1:::rbd_data.11362e0d5ebf.000000000000001b:head# 2024-09-16T14:09:30.231 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2c7a6f1:::rbd_data.1136f7114f14.0000000000000088:18# 2024-09-16T14:09:30.231 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2c7a6f1:::rbd_data.1136f7114f14.0000000000000088:head# 2024-09-16T14:09:30.231 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2da8132:::rbd_data.113612ae6a73.0000000000000012:head# 2024-09-16T14:09:30.232 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2ec89bc:::rbd_data.11361fa8f1cb.0000000000000128:head# 2024-09-16T14:09:30.232 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2f03c90:::rbd_data.11362e0d5ebf.000000000000008c:head# 2024-09-16T14:09:30.232 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2f05e7e:::rbd_object_map.11366c63f9a8.000000000000002e:head# 2024-09-16T14:09:30.233 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c2fa6904:::rbd_data.11361fa8f1cb.000000000000007f:head# 2024-09-16T14:09:30.233 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c311bb18:::rbd_data.113612ae6a73.0000000000000013:head# 2024-09-16T14:09:30.233 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c33258e3:::rbd_data.11361fa8f1cb.00000000000000ad:head# 2024-09-16T14:09:30.233 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c3374f04:::rbd_data.1136f7114f14.0000000000000039:18# 2024-09-16T14:09:30.233 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c3374f04:::rbd_data.1136f7114f14.0000000000000039:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c351b7b4:::rbd_data.11367eb52311.0000000000000267:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c3a55820:::rbd_data.11363617f81b.0000000000000025:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c3a9b7e3:::rbd_data.11363617f81b.000000000000000a:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c3e54240:::rbd_data.11361fa8f1cb.0000000000000188:d# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c3e54240:::rbd_data.11361fa8f1cb.0000000000000188:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c3f34fb1:::rbd_data.113618577e70.000000000000003a:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c40097e4:::rbd_id.image_client.0-clone37:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c438de0c:::rbd_data.11361fa8f1cb.000000000000007b:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c448f0d0:::rbd_data.11361fa8f1cb.0000000000000152:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c44edef7:::rbd_data.11361fa8f1cb.000000000000013b:head# 2024-09-16T14:09:30.235 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c47e40d5:::rbd_data.11361fa8f1cb.0000000000000127:head# 2024-09-16T14:09:30.236 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c4b84924:::rbd_data.11362defb50e.00000000000000ac:head# 2024-09-16T14:09:30.236 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c4c91d4e:::rbd_data.11361fa8f1cb.00000000000000c3:head# 2024-09-16T14:09:30.236 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c4c92cc0:::rbd_data.113631a28a63.0000000000000003:head# 2024-09-16T14:09:30.238 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c503cd86:::rbd_data.11363617f81b.0000000000000076:head# 2024-09-16T14:09:30.238 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5040228:::rbd_data.11363617f81b.000000000000007e:head# 2024-09-16T14:09:30.238 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5192c5c:::rbd_data.1136f033099c.0000000000000018:head# 2024-09-16T14:09:30.238 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c54e9e1e:::rbd_object_map.1136edad242a:head# 2024-09-16T14:09:30.238 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c55f55b1:::rbd_data.113618577e70.0000000000000007:head# 2024-09-16T14:09:30.239 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c56f5f0c:::rbd_data.11362e0d5ebf.0000000000000000:head# 2024-09-16T14:09:30.239 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5757882:::rbd_data.1136494ba971.000000000000002c:7# 2024-09-16T14:09:30.239 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5757882:::rbd_data.1136494ba971.000000000000002c:head# 2024-09-16T14:09:30.239 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c57dba49:::rbd_header.1136f306a5c5:head# 2024-09-16T14:09:30.240 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c57ecfe3:::rbd_data.1136e112ce26.000000000000000a:head# 2024-09-16T14:09:30.240 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c583a80b:::rbd_data.11361fa8f1cb.00000000000000eb:head# 2024-09-16T14:09:30.240 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5a01ce0:::rbd_data.1136e112ce26.0000000000000011:head# 2024-09-16T14:09:30.240 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5b16710:::rbd_data.11362defb50e.0000000000000022:head# 2024-09-16T14:09:30.241 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5bc4a13:::rbd_data.11367eb52311.000000000000005c:head# 2024-09-16T14:09:30.241 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7# 2024-09-16T14:09:30.241 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:head# 2024-09-16T14:09:30.241 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c5e532f9:::rbd_data.113618577e70.0000000000000025:head# 2024-09-16T14:09:30.241 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c60e55d0:::rbd_data.11368cf3b67c.0000000000000013:1b# 2024-09-16T14:09:30.242 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c60e55d0:::rbd_data.11368cf3b67c.0000000000000013:head# 2024-09-16T14:09:30.243 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c67b9c3d:::rbd_data.1136494ba971.0000000000000059:7# 2024-09-16T14:09:30.243 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c67b9c3d:::rbd_data.1136494ba971.0000000000000059:head# 2024-09-16T14:09:30.243 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c681cb6b:::rbd_data.11361fa8f1cb.00000000000000ae:head# 2024-09-16T14:09:30.243 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c6a0df78:::rbd_data.1136f7114f14.000000000000005d:head# 2024-09-16T14:09:30.243 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c6bf73b2:::rbd_data.11369618fede.0000000000000042:head# 2024-09-16T14:09:30.243 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c6c58a86:::rbd_data.11367eb52311.000000000000000a:head# 2024-09-16T14:09:30.243 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c6d65799:::rbd_data.11362e0d5ebf.0000000000000016:head# 2024-09-16T14:09:30.244 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c727c263:::rbd_data.1136f7114f14.000000000000003c:18# 2024-09-16T14:09:30.244 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c727c263:::rbd_data.1136f7114f14.000000000000003c:head# 2024-09-16T14:09:30.244 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c73e5e54:::rbd_data.1136494ba971.000000000000005c:7# 2024-09-16T14:09:30.245 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c73e5e54:::rbd_data.1136494ba971.000000000000005c:head# 2024-09-16T14:09:30.245 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c741a2c9:::rbd_data.11362defb50e.00000000000000b0:head# 2024-09-16T14:09:30.245 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c7579591:::rbd_data.11369af4c40c.0000000000000019:8# 2024-09-16T14:09:30.247 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c7579591:::rbd_data.11369af4c40c.0000000000000019:head# 2024-09-16T14:09:30.247 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c7af227e:::rbd_data.113619b3ee73.0000000000000022:head# 2024-09-16T14:09:30.247 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c7b8a828:::rbd_data.11363617f81b.0000000000000094:head# 2024-09-16T14:09:30.248 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c7b9ff27:::rbd_data.11361fa8f1cb.0000000000000074:head# 2024-09-16T14:09:30.248 INFO:teuthology.orchestra.run.smithi079.stderr:Read #3:c7f1fd7d:::rbd_data.11368b74a150.000000000000004c:head# 2024-09-16T14:09:30.248 INFO:teuthology.orchestra.run.smithi079.stderr:Export successful 2024-09-16T14:09:30.282 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:30.282 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:30.283 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:30.279+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:30.291 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:30.385 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:30.381+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:30.401 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:30.402 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:30.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:30.485 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:30.479+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:30.508 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:30.508 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:30.517 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:30.587 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:30.583+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:30.613 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:30.613 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:30.622 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:30.688 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:30.683+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:30.716 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:30.716 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:30.726 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:30.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.3 2024-09-16T14:09:30.788 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:30.785+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:30.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:09:30.830 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:30.830 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:30.839 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:30.889 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:30.885+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:30.990 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:30.985+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:31.001 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:31.001 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:31.010 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:31.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-16T14:09:31.091 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:31.089+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:31.107 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:31.107 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:31.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:31.192 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:31.187+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:31.221 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:31.221 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:31.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:31.293 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:31.289+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:31.328 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:31.328 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:31.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:31.394 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:31.389+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:31.433 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:31.433 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:31.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:31.495 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:31.493+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:31.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:31.550 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:31.551 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:31.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:31.606 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:31.603+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:31.649 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:31.667 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:31.689 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:31.697 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:31.693+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:31.706 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:31.706 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:31.707 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:31.710 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:31.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:31.730 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:31.768 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:31.797 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:31.791+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:31.801 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:31.801 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:31.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:31.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:31.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:31.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:31.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:31.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:31.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:31.806 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:31.807 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.930609097000001, 2024-09-16T14:09:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:09:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:09:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.930586388999998, 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:09:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:09:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.930562237000004, 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:09:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.930529296000003, 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:09:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:09:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.930504612999997, 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:09:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:09:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.930481384000004, 2024-09-16T14:09:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:09:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:09:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.928635420000006, 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:09:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.928592555999998, 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:09:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927132938, 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:09:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927117168999999, 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:09:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:09:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:09:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927101417000003, 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:09:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927085985999994, 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:09:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927070348000001, 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:09:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:09:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927055209000002, 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:09:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927040371999993, 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:09:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:09:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927017367999994, 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:09:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:09:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.927002142999996, 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:09:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:09:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.926986959000004, 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:09:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.926971714000004, 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:09:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.926956555999993, 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:09:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:09:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:09:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:31.948 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:31.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:31.950 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:31.945+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: marking collection for removal 2024-09-16T14:09:31.989 INFO:teuthology.orchestra.run.smithi079.stdout:setting '_remove' omap key 2024-09-16T14:09:31.995 INFO:teuthology.orchestra.run.smithi079.stdout:finish_remove_pgs 3.3_head removing 3.3 2024-09-16T14:09:32.017 INFO:teuthology.orchestra.run.smithi079.stdout:Remove successful 2024-09-16T14:09:32.050 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:32.047+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:32.090 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:32.091 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:32.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:32.151 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:32.147+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:32.246 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:32.247 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:32.247 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:32.251 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:32.249+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:32.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:32.352 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:32.347+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:32.359 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:32.359 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:32.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:32.359 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:32.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.359 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:09:32.359 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.836539545999997, 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:09:32.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 50.831268207000001, 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:32.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:09:32.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.648759693000002, 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.372 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:09:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.645359161999998, 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.377 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:09:32.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:09:32.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.641307883000003, 2024-09-16T14:09:32.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:09:32.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.637554967, 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.390 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.633475439000001, 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:09:32.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:09:32.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.633458836999999, 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:09:32.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.633442860999999, 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:32.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:09:32.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.633424908999999, 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.404 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:09:32.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.405 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:32.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:09:32.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.629229778000003, 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.413 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.629132800000001, 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:09:32.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.629108258999999, 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.625551497000004, 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:09:32.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.625533738999998, 2024-09-16T14:09:32.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:09:32.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:09:32.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.625516075999997, 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:09:32.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:09:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.625498882000002, 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:09:32.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.621802463999998, 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:32.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.609472511999996, 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.445 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.449 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 17.935960340000001, 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:09:32.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:09:32.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:32.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:32.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:32.456 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:32.458 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:32.458 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:32.451+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:32.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.3.0 2024-09-16T14:09:32.559 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:32.555+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:32.569 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:32.569 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:32.569 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:32.569 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:32.578 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:32.660 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:32.657+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:32.681 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:32.681 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:32.681 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:32.681 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:32.681 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:32.690 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:32.761 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:32.755+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:32.791 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:32.791 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:32.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:32.791 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:32.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.449871559, 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:09:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:09:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:09:32.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.44984916199999, 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.798 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:09:32.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.449242451, 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:09:32.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:09:32.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:32.810 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.44916812400001, 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:32.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:09:32.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.816 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.47824333200001, 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:09:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.850284751, 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:09:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.32012428500001, 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:09:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:09:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.672575753999993, 2024-09-16T14:09:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:09:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:09:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101795 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 174080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.126210630000003, 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020037534999999999, 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101795 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660560+0000", 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9699999999999996e-07 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660562+0000", 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0839999999999999e-06 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660568+0000", 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0480000000000004e-06 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660573+0000", 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1239999999999998e-06 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680395+0000", 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019822487 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680488+0000", 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2601000000000003e-05 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680597+0000", 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109694 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.126168449999994, 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:09:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.126134225000001, 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:09:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.126104509000001, 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:09:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:09:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:09:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.126084847000001, 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:09:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.117324722000006, 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:09:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:09:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.032598179999994, 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:09:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:09:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.387594643, 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:09:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:09:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:09:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.387507782, 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:09:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.198201163, 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.312976940999999, 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:09:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.295913759000001, 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:09:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:09:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:32.903 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:32.904 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:32.905 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:32.899+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:33.005 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:33.001+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:33.015 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:33.015 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:33.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.015 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:194462 3.1a e141/125 3:59937af8:::rbd_header.1136b1466cf:head v 141'683, mlcod=141'681)", 2024-09-16T14:09:33.015 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:33.013892+0000", 2024-09-16T14:09:33.015 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00028769499999999998, 2024-09-16T14:09:33.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032551100000000001, 2024-09-16T14:09:33.015 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.013892+0000", 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.013892+0000", 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.013893+0000", 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6470000000000001e-06 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.013911+0000", 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8159e-05 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.013941+0000", 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9580000000000001e-05 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.013952+0000", 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0786e-05 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.014013+0000", 2024-09-16T14:09:33.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1716000000000003e-05 2024-09-16T14:09:33.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.014037+0000", 2024-09-16T14:09:33.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3572999999999999e-05 2024-09-16T14:09:33.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.020 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.020 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:09:33.020 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T14:09:33.020 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:33.025 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:33.106 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:33.101+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:33.121 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:33.121 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:33.121 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:33.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:33.207 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:33.203+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:33.235 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:33.235 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:33.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:33.235 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:33.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.235 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.235 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.400626809000002, 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:33.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:33.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.400610470000004, 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:33.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.400585305000007, 2024-09-16T14:09:33.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:33.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:33.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.399681337999994, 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:33.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.257 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.399671999000006, 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:33.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:33.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:33.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.399381876999996, 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:33.264 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:33.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:33.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.269 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.399163182999999, 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:33.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.392769603999994, 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:33.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:33.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:33.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.388858041000006, 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:33.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:33.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.285 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.388826836000007, 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:33.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.388007396000006, 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:33.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:33.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:33.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.387962587000004, 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:33.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.384953332999999, 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:33.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:33.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.384936373000002, 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:33.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:33.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.384924877000003, 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:33.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.380177966999995, 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:33.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.372545336000002, 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:33.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:33.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.372533129000004, 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.329 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.370278213999995, 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:33.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:33.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:33.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.370053963000004, 2024-09-16T14:09:33.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:33.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.345 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:33.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:33.347 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:33.343+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:33.447 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:33.443+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:33.449 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:33.449 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:33.449 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:33.449 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:33.458 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:33.548 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:33.545+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:33.557 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:33.557 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:33.566 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:33.649 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:33.643+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:33.670 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:33.670 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:33.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:33.670 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:33.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.291665967, 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.287734549000007, 2024-09-16T14:09:33.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.679 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.287710356999995, 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:09:33.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:09:33.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:09:33.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:33.684 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.283991086, 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:33.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.280074321000001, 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.689 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:09:33.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.275692042000003, 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:09:33.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.275668201000002, 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:09:33.698 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:09:33.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:09:33.702 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.275653012999996, 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:09:33.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.707 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.275638427000004, 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:33.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:33.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:09:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.712 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.275534777000004, 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:33.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.717 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.275507051000005, 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:09:33.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.272806720999995, 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.726 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.272766554, 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.727 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.272676838999999, 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:09:33.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:09:33.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.272167851000006, 2024-09-16T14:09:33.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:09:33.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.264070481000005, 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:09:33.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:09:33.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:09:33.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 85.261902188999997, 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:09:33.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:09:33.751 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.112340728, 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:09:33.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:09:33.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:09:33.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [watch reconnect cookie 140383155395456 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.982331500000001, 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.013377462, 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:33.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:09:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:09:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:09:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:09:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:09:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:09:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:09:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:09:33.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:09:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:09:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:09:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:09:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:09:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:09:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:09:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:33.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:09:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:09:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.694814+0000", 2024-09-16T14:09:33.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003026674 2024-09-16T14:09:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.695138+0000", 2024-09-16T14:09:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032383800000000001 2024-09-16T14:09:33.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697164+0000", 2024-09-16T14:09:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020260149999999999 2024-09-16T14:09:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.976468366000001, 2024-09-16T14:09:33.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.007999884, 2024-09-16T14:09:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:09:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:33.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:09:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:09:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:33.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:09:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:09:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:09:33.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:09:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:09:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:09:33.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:09:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:09:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:09:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:09:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:33.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696135+0000", 2024-09-16T14:09:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003762636 2024-09-16T14:09:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696215+0000", 2024-09-16T14:09:33.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0119999999999999e-05 2024-09-16T14:09:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697528+0000", 2024-09-16T14:09:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001312887 2024-09-16T14:09:33.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697582+0000", 2024-09-16T14:09:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4409e-05 2024-09-16T14:09:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:33.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697649+0000", 2024-09-16T14:09:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7026000000000005e-05 2024-09-16T14:09:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:33.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:33.785 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:33.788 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:33.805 INFO:teuthology.orchestra.run.smithi079.stdout:get_pg_num_history pg_num_history pg_num_history(e129 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-16T14:09:33.805 INFO:teuthology.orchestra.run.smithi079.stdout:Importing pgid 3.3 2024-09-16T14:09:33.805 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0101559:::rbd_data.11361fa8f1cb.000000000000001f:head# 2024-09-16T14:09:33.805 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.806 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0140dcc:::rbd_data.11364e09a271.0000000000000002:30# 2024-09-16T14:09:33.806 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0140dcc:::rbd_data.11364e09a271.0000000000000002:head# 2024-09-16T14:09:33.806 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 30=[]:{30=[30]} 2024-09-16T14:09:33.806 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0231641:::rbd_data.1136f033099c.0000000000000032:head# 2024-09-16T14:09:33.806 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 2b=[]:{} 2024-09-16T14:09:33.806 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c04552a4:::rbd_object_map.1136dd12cea0:head# 2024-09-16T14:09:33.806 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.807 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c04959d7:::rbd_data.113630ceda18.0000000000000073:25# 2024-09-16T14:09:33.807 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c04959d7:::rbd_data.113630ceda18.0000000000000073:head# 2024-09-16T14:09:33.807 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 25=[]:{25=[25]} 2024-09-16T14:09:33.807 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0783ec2:::rbd_data.11361fa8f1cb.0000000000000076:head# 2024-09-16T14:09:33.807 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.807 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0977406:::rbd_data.1136494ba971.000000000000000f:7# 2024-09-16T14:09:33.807 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0977406:::rbd_data.1136494ba971.000000000000000f:head# 2024-09-16T14:09:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 7=[]:{7=[7]} 2024-09-16T14:09:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0c9ab1e:::rbd_data.113677f46bcd.0000000000000008:27# 2024-09-16T14:09:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0c9ab1e:::rbd_data.113677f46bcd.0000000000000008:head# 2024-09-16T14:09:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 27=[]:{27=[27]} 2024-09-16T14:09:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0d084e0:::rbd_data.11361fa8f1cb.000000000000002b:head# 2024-09-16T14:09:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.808 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0d782ac:::rbd_data.1136a570af09.0000000000000001:head# 2024-09-16T14:09:33.809 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.809 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0e631c4:::rbd_header.1136494ba971:head# 2024-09-16T14:09:33.809 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.809 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0ea02dd:::rbd_data.113630ceda18.0000000000000002:head# 2024-09-16T14:09:33.809 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.809 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c0f16a19:::rbd_data.1136f033099c.0000000000000000:head# 2024-09-16T14:09:33.809 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.809 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c10e1f6a:::rbd_data.11362e0d5ebf.0000000000000075:head# 2024-09-16T14:09:33.810 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.810 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c12cb214:::rbd_data.11363617f81b.00000000000000dc:head# 2024-09-16T14:09:33.810 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 1f=[]:{} 2024-09-16T14:09:33.810 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1527b88:::rbd_data.11367eb52311.0000000000000134:head# 2024-09-16T14:09:33.810 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.810 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c16144c7:::rbd_data.11369618fede.000000000000000d:head# 2024-09-16T14:09:33.810 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.811 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1b850a0:::rbd_data.11362defb50e.000000000000009c:head# 2024-09-16T14:09:33.811 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.811 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1c67fb6:::rbd_data.11362e0d5ebf.0000000000000006:head# 2024-09-16T14:09:33.811 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.811 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1c80357:::rbd_data.113612ae6a73.0000000000000016:13# 2024-09-16T14:09:33.811 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1c80357:::rbd_data.113612ae6a73.0000000000000016:head# 2024-09-16T14:09:33.811 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 13=[]:{13=[13]} 2024-09-16T14:09:33.811 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1d2593c:::rbd_data.11368cf3b67c.0000000000000008:1b# 2024-09-16T14:09:33.812 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1d2593c:::rbd_data.11368cf3b67c.0000000000000008:head# 2024-09-16T14:09:33.812 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-16T14:09:33.812 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1d2c68b:::rbd_data.11361fa8f1cb.00000000000000f4:head# 2024-09-16T14:09:33.812 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.812 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1ecc080:::rbd_data.113619b3ee73.0000000000000041:16# 2024-09-16T14:09:33.812 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1ecc080:::rbd_data.113619b3ee73.0000000000000041:head# 2024-09-16T14:09:33.812 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 16=[]:{16=[16]} 2024-09-16T14:09:33.813 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c1eec317:::rbd_data.11369618fede.0000000000000014:head# 2024-09-16T14:09:33.813 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.813 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c203ff01:::rbd_data.11361fb616f9.000000000000000a:14# 2024-09-16T14:09:33.813 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c203ff01:::rbd_data.11361fb616f9.000000000000000a:head# 2024-09-16T14:09:33.813 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 14=[]:{14=[14]} 2024-09-16T14:09:33.813 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2172984:::rbd_object_map.1136a570af09:head# 2024-09-16T14:09:33.813 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.814 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c259c83c:::rbd_data.113699bfa25d.0000000000000014:head# 2024-09-16T14:09:33.814 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.814 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c26647c7:::rbd_data.11361fa8f1cb.0000000000000178:head# 2024-09-16T14:09:33.814 INFO:teuthology.orchestra.run.smithi079.stdout:snapset d=[]:{} 2024-09-16T14:09:33.814 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c267fc67:::rbd_data.113619b3ee73.000000000000003d:16# 2024-09-16T14:09:33.814 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c267fc67:::rbd_data.113619b3ee73.000000000000003d:head# 2024-09-16T14:09:33.815 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 16=[]:{16=[16]} 2024-09-16T14:09:33.815 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c26990c5:::rbd_data.11361fa8f1cb.00000000000000e7:head# 2024-09-16T14:09:33.815 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.815 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2b8aec4:::rbd_header.1136a719415e:head# 2024-09-16T14:09:33.815 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.815 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2c37deb:::rbd_data.11369618fede.0000000000000015:head# 2024-09-16T14:09:33.815 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.815 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2c5f3b1:::rbd_data.11362e0d5ebf.000000000000001b:head# 2024-09-16T14:09:33.816 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.816 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2c7a6f1:::rbd_data.1136f7114f14.0000000000000088:18# 2024-09-16T14:09:33.816 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2c7a6f1:::rbd_data.1136f7114f14.0000000000000088:head# 2024-09-16T14:09:33.816 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 18=[]:{18=[18]} 2024-09-16T14:09:33.816 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2da8132:::rbd_data.113612ae6a73.0000000000000012:head# 2024-09-16T14:09:33.816 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 13=[]:{} 2024-09-16T14:09:33.816 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2ec89bc:::rbd_data.11361fa8f1cb.0000000000000128:head# 2024-09-16T14:09:33.817 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.817 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2f03c90:::rbd_data.11362e0d5ebf.000000000000008c:head# 2024-09-16T14:09:33.817 INFO:teuthology.orchestra.run.smithi079.stdout:snapset a=[]:{} 2024-09-16T14:09:33.817 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2f05e7e:::rbd_object_map.11366c63f9a8.000000000000002e:head# 2024-09-16T14:09:33.817 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.817 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c2fa6904:::rbd_data.11361fa8f1cb.000000000000007f:head# 2024-09-16T14:09:33.817 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.817 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c311bb18:::rbd_data.113612ae6a73.0000000000000013:head# 2024-09-16T14:09:33.818 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 13=[]:{} 2024-09-16T14:09:33.818 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c33258e3:::rbd_data.11361fa8f1cb.00000000000000ad:head# 2024-09-16T14:09:33.818 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.818 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c3374f04:::rbd_data.1136f7114f14.0000000000000039:18# 2024-09-16T14:09:33.818 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c3374f04:::rbd_data.1136f7114f14.0000000000000039:head# 2024-09-16T14:09:33.818 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 18=[]:{18=[18]} 2024-09-16T14:09:33.818 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c351b7b4:::rbd_data.11367eb52311.0000000000000267:head# 2024-09-16T14:09:33.818 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.819 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c3a55820:::rbd_data.11363617f81b.0000000000000025:head# 2024-09-16T14:09:33.819 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.819 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c3a9b7e3:::rbd_data.11363617f81b.000000000000000a:head# 2024-09-16T14:09:33.819 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.819 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c3e54240:::rbd_data.11361fa8f1cb.0000000000000188:d# 2024-09-16T14:09:33.819 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c3e54240:::rbd_data.11361fa8f1cb.0000000000000188:head# 2024-09-16T14:09:33.819 INFO:teuthology.orchestra.run.smithi079.stdout:snapset d=[]:{d=[d]} 2024-09-16T14:09:33.820 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c3f34fb1:::rbd_data.113618577e70.000000000000003a:head# 2024-09-16T14:09:33.820 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 17=[]:{} 2024-09-16T14:09:33.820 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c40097e4:::rbd_id.image_client.0-clone37:head# 2024-09-16T14:09:33.820 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.820 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c438de0c:::rbd_data.11361fa8f1cb.000000000000007b:head# 2024-09-16T14:09:33.820 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.820 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c448f0d0:::rbd_data.11361fa8f1cb.0000000000000152:head# 2024-09-16T14:09:33.820 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.821 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c44edef7:::rbd_data.11361fa8f1cb.000000000000013b:head# 2024-09-16T14:09:33.821 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.821 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c47e40d5:::rbd_data.11361fa8f1cb.0000000000000127:head# 2024-09-16T14:09:33.821 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.821 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c4b84924:::rbd_data.11362defb50e.00000000000000ac:head# 2024-09-16T14:09:33.821 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.821 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c4c91d4e:::rbd_data.11361fa8f1cb.00000000000000c3:head# 2024-09-16T14:09:33.822 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.822 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c4c92cc0:::rbd_data.113631a28a63.0000000000000003:head# 2024-09-16T14:09:33.822 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.822 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c503cd86:::rbd_data.11363617f81b.0000000000000076:head# 2024-09-16T14:09:33.822 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.822 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5040228:::rbd_data.11363617f81b.000000000000007e:head# 2024-09-16T14:09:33.822 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.822 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5192c5c:::rbd_data.1136f033099c.0000000000000018:head# 2024-09-16T14:09:33.823 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.823 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c54e9e1e:::rbd_object_map.1136edad242a:head# 2024-09-16T14:09:33.823 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.823 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c55f55b1:::rbd_data.113618577e70.0000000000000007:head# 2024-09-16T14:09:33.823 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 17=[]:{} 2024-09-16T14:09:33.823 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c56f5f0c:::rbd_data.11362e0d5ebf.0000000000000000:head# 2024-09-16T14:09:33.823 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.824 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:33.783+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:33.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:33.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:09:33.828 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5757882:::rbd_data.1136494ba971.000000000000002c:7# 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5757882:::rbd_data.1136494ba971.000000000000002c:head# 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 7=[]:{7=[7]} 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c57dba49:::rbd_header.1136f306a5c5:head# 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c57ecfe3:::rbd_data.1136e112ce26.000000000000000a:head# 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 1c=[]:{} 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c583a80b:::rbd_data.11361fa8f1cb.00000000000000eb:head# 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5a01ce0:::rbd_data.1136e112ce26.0000000000000011:head# 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 1c=[]:{} 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5b16710:::rbd_data.11362defb50e.0000000000000022:head# 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.829 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5bc4a13:::rbd_data.11367eb52311.000000000000005c:head# 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:7# 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5d6fbb5:::rbd_data.1136494ba971.000000000000000a:head# 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 7=[]:{7=[7]} 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c5e532f9:::rbd_data.113618577e70.0000000000000025:head# 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 17=[]:{} 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c60e55d0:::rbd_data.11368cf3b67c.0000000000000013:1b# 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c60e55d0:::rbd_data.11368cf3b67c.0000000000000013:head# 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c67b9c3d:::rbd_data.1136494ba971.0000000000000059:7# 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c67b9c3d:::rbd_data.1136494ba971.0000000000000059:head# 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 7=[]:{7=[7]} 2024-09-16T14:09:33.830 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c681cb6b:::rbd_data.11361fa8f1cb.00000000000000ae:head# 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c6a0df78:::rbd_data.1136f7114f14.000000000000005d:head# 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 18=[]:{} 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c6bf73b2:::rbd_data.11369618fede.0000000000000042:head# 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c6c58a86:::rbd_data.11367eb52311.000000000000000a:head# 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c6d65799:::rbd_data.11362e0d5ebf.0000000000000016:head# 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c727c263:::rbd_data.1136f7114f14.000000000000003c:18# 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c727c263:::rbd_data.1136f7114f14.000000000000003c:head# 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 18=[]:{18=[18]} 2024-09-16T14:09:33.831 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c73e5e54:::rbd_data.1136494ba971.000000000000005c:7# 2024-09-16T14:09:33.832 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c73e5e54:::rbd_data.1136494ba971.000000000000005c:head# 2024-09-16T14:09:33.832 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 7=[]:{7=[7]} 2024-09-16T14:09:33.832 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c741a2c9:::rbd_data.11362defb50e.00000000000000b0:head# 2024-09-16T14:09:33.832 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.832 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c7579591:::rbd_data.11369af4c40c.0000000000000019:8# 2024-09-16T14:09:33.832 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c7579591:::rbd_data.11369af4c40c.0000000000000019:head# 2024-09-16T14:09:33.832 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 8=[]:{8=[8]} 2024-09-16T14:09:33.833 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c7af227e:::rbd_data.113619b3ee73.0000000000000022:head# 2024-09-16T14:09:33.833 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 16=[]:{} 2024-09-16T14:09:33.833 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c7b8a828:::rbd_data.11363617f81b.0000000000000094:head# 2024-09-16T14:09:33.834 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.834 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c7b9ff27:::rbd_data.11361fa8f1cb.0000000000000074:head# 2024-09-16T14:09:33.835 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.835 INFO:teuthology.orchestra.run.smithi079.stdout:Write #3:c7f1fd7d:::rbd_data.11368b74a150.000000000000004c:head# 2024-09-16T14:09:33.835 INFO:teuthology.orchestra.run.smithi079.stdout:snapset 0=[]:{} 2024-09-16T14:09:33.835 INFO:teuthology.orchestra.run.smithi079.stdout:write_pg epoch 121 info 3.3( v 121'978 (0'0,121'978] local-lis/les=23/24 n=104 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-16T14:09:33.841 INFO:teuthology.orchestra.run.smithi079.stdout:Import successful 2024-09-16T14:09:33.887 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:33.885+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:33.908 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:33.908 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:33.919 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:33.953 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:33.973 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:33.988 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:33.985+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:33.993 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:34.013 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:34.041 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:34.060 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:34.069 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:34.069 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:34.069 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:34.069 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:34.069 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:34.079 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:34.080 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:34.080 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:34.080 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:34.080 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:34.080 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:34.081 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:34.081 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:34.081 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:34.081 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:34.084 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:34.084 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:34.089 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:34.085+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:34.173 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:34.173 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:34.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:34.227 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:34.225+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:34.285 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:34.285 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:34.292 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:34.287+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:34.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:34.385 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /var/log/ceph/exp.3.3.0 2024-09-16T14:09:34.392 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:34.389+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:34.394 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:34.394 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:34.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:34.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:09:34.493 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:34.487+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:34.510 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:34.510 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:34.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:34.594 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:34.589+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:34.658 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:34.658 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:34.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:34.694 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:34.693+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:34.707 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:09:34.708 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":142,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:09:33.812441+0000","last_up_change":"2024-09-16T14:09:18.679093+0000","last_in_change":"2024-09-16T14:09:13.363709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:06:58.526935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T14:07:08.375919+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":60,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":130,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":22196},{"type":"v1","addr":"172.21.15.79:6817","nonce":22196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":22196},{"type":"v1","addr":"172.21.15.79:6819","nonce":22196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":22196},{"type":"v1","addr":"172.21.15.79:6824","nonce":22196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":22196},{"type":"v1","addr":"172.21.15.79:6821","nonce":22196}]},"public_addr":"172.21.15.79:6817/22196","cluster_addr":"172.21.15.79:6819/22196","heartbeat_back_addr":"172.21.15.79:6824/22196","heartbeat_front_addr":"172.21.15.79:6821/22196","state":["exists"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":83,"up_thru":88,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":35927},{"type":"v1","addr":"172.21.15.79:6801","nonce":35927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":35927},{"type":"v1","addr":"172.21.15.79:6803","nonce":35927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":35927},{"type":"v1","addr":"172.21.15.79:6807","nonce":35927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":35927},{"type":"v1","addr":"172.21.15.79:6805","nonce":35927}]},"public_addr":"172.21.15.79:6801/35927","cluster_addr":"172.21.15.79:6803/35927","heartbeat_back_addr":"172.21.15.79:6807/35927","heartbeat_front_addr":"172.21.15.79:6805/35927","state":["exists","up"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":0.94293212890625,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":130},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":59},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":47,"length":7}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":47,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:09:34.722 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T14:06:52.822826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T14:06:58.526935+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T14:07:08.375919+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 60, 'snap_epoch': 142, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-09-16T14:09:34.722 DEBUG:teuthology.orchestra.run.smithi079:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool .mgr 2024-09-16T14:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:34.765 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:34.774 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:34.795 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:34.793+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:34.841 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:09:34.878 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:34.879 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:34.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:34.896 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:34.893+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:34.997 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:34.993+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:35.043 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.043 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:35.098 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:35.097+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:35.147 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.147 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:35.199 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:35.195+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:35.257 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.258 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:35.310 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:35.307+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:35.376 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.377 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:35.401 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:35.397+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:35.485 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.485 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:35.539 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:35.537+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:35.595 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.595 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.602 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:35.599+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:35.604 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:35.701 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.701 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.703 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:35.701+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:35.710 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:35.804 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:35.799+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:35.809 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.809 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.818 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:35.870 INFO:teuthology.orchestra.run.smithi079.stderr:Nothing to do for non-filestore backend 2024-09-16T14:09:35.905 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:35.901+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:35.923 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:35.923 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:35.932 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:36.006 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:36.003+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:36.025 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:36.025 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:36.034 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:36.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:36.107 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:36.105+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:36.129 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:36.130 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:36.138 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:36.208 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:36.205+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:36.231 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:36.231 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:36.231 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:36.239 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:36.251 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:36.273 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:36.296 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:36.309 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:36.303+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:36.318 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:36.339 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:36.361 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:36.361 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:36.361 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:36.361 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:36.361 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:36.362 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:36.362 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:36.362 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:36.362 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:36.365 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:36.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:36.382 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:36.382 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:36.382 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:36.383 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:36.392 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:36.410 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:36.405+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:36.422 DEBUG:teuthology.orchestra.run.smithi079:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool rbd 2024-09-16T14:09:36.488 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:36.489 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:36.489 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:36.489 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:36.489 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:36.497 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:36.511 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:36.507+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:36.598 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:36.598 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.18556263599999, 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:36.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:36.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.347600313, 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:36.605 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:36.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:36.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.820361324999993, 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:36.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:36.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.820338093999993, 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:36.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.820315147000002, 2024-09-16T14:09:36.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:36.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:36.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.820298637999997, 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:36.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:36.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:36.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.629 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.820281011999995, 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.630 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:36.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:36.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.814365268000003, 2024-09-16T14:09:36.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:36.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.814347862999995, 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:36.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.813993496999998, 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.644 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:36.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:36.648 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.810849637999993, 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:36.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:36.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.810834352000001, 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:36.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.657 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.810817943000004, 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:36.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.662 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.810742746000003, 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:36.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:36.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:36.667 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.810645199000007, 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:36.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:36.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:36.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:36.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:36.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:36.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.810619958000004, 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:36.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.810590903999994, 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.679 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:36.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:36.684 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.807326915000004, 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:36.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.807308714000001, 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.690 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:36.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.694 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.807269536000007, 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:36.695 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:36.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:36.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:36.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:36.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:36.700 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:36.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:36.702 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:36.697+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:36.803 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:36.801+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:36.836 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:36.836 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:36.837 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:36.837 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:36.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:36.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:09:36.903 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:36.899+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:36.949 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:36.949 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:36.949 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:36.949 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:36.949 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:36.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:37.004 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:37.001+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:37.105 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:37.101+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:37.110 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:37.110 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:37.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:37.110 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:37.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.110 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:37.110 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.870797738999997, 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:37.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:37.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510644696, 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:37.115 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:37.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.119 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510635696999998, 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:37.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:37.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510627346, 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:37.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510594662000003, 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:37.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.133 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510585425000002, 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:37.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:37.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:37.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:37.138 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510576893, 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.139 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:37.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:37.143 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510568315999997, 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:37.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510558955, 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:37.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:37.152 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510550842999997, 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:37.156 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510539139999999, 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.157 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:37.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510530527, 2024-09-16T14:09:37.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:37.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:37.165 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510522420999997, 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.166 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510514079000004, 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:37.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:37.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510505541000001, 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:37.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510497209, 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:37.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:37.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510488967000001, 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.185 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:37.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510479404000002, 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:37.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:37.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510470023000003, 2024-09-16T14:09:37.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:37.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.510460381999998, 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.199 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:37.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.203 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:37.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:37.206 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:37.201+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:37.307 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:37.307 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:37.307 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:37.307 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:37.303+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:37.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:37.605 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:37.606 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:37.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:37.607 INFO:teuthology.orchestra.run.smithi079.stderr:Nothing to do for non-filestore backend 2024-09-16T14:09:37.608 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:37.605+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:37.708 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:37.705+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.805705432, 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:09:37.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:09:37.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.80568272399999, 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.720 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:09:37.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.805658572, 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:09:37.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.805625631, 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.732 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:37.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:09:37.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.80560094800001, 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:09:37.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:09:37.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:09:37.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.743 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.805577719, 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:37.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.803731755, 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.750 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:37.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.80368889099999, 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:09:37.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.80222927299999, 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:09:37.761 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:09:37.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:09:37.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:09:37.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:09:37.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.80221350399999, 2024-09-16T14:09:37.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:09:37.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802197752, 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:37.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:09:37.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802182321, 2024-09-16T14:09:37.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:09:37.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802166683, 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:09:37.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802151544, 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:09:37.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:09:37.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:37.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:37.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:09:37.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:09:37.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:09:37.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:09:37.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:09:37.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:09:37.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:09:37.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:09:37.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:09:37.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:37.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:09:37.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:09:37.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.809 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.809 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:37.809 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802136707, 2024-09-16T14:09:37.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:09:37.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.810 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:09:37.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.810 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:37.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:09:37.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802113703, 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:37.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802098478, 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:09:37.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:09:37.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:09:37.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802083294, 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.826 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:09:37.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:09:37.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802068049, 2024-09-16T14:09:37.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:09:37.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:09:37.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 103.802052891, 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.842 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:37.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:37.844 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:37.839+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:37.872 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:09:37.944 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:37.939+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:37.977 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:195332 3.3 3:c0f16a19:::rbd_data.1136f033099c.0000000000000000:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-16T14:09:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:37.973279+0000", 2024-09-16T14:09:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0024926779999999999, 2024-09-16T14:09:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025271930000000001, 2024-09-16T14:09:37.978 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 195332 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973279+0000", 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973279+0000", 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973280+0000", 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0750000000000001e-06 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973285+0000", 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.609e-06 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973298+0000", 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2554e-05 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973307+0000", 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7700000000000007e-06 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973423+0000", 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000116057 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973668+0000", 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024509400000000001 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:195341 3.13 3:c891cbbe:::rbd_data.1136f033099c.0000000000000009:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:37.973413+0000", 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.0023585059999999998, 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002495897, 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 195341 2024-09-16T14:09:37.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973413+0000", 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973413+0000", 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973414+0000", 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.356e-06 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973420+0000", 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3969999999999999e-06 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973437+0000", 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7025e-05 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973443+0000", 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.857e-06 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973527+0000", 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4096000000000003e-05 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.973735+0000", 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020744200000000001 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975813+0000", 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020784549999999999 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975867+0000", 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3417e-05 2024-09-16T14:09:37.987 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:195336 3.0 e146/95 3:056b1641:::rbd_data.1136f033099c.0000000000000004:head v 146'1189, mlcod=145'1187)", 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:37.974295+0000", 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.001476223, 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017296410000000001, 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974295+0000", 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974295+0000", 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974296+0000", 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8100000000000001e-07 2024-09-16T14:09:37.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974303+0000", 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5590000000000002e-06 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974315+0000", 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2157999999999999e-05 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974320+0000", 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1089999999999997e-06 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974376+0000", 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6520000000000001e-05 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974407+0000", 2024-09-16T14:09:37.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0898000000000001e-05 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:195335 3.1 e146/23 3:865cb3cc:::rbd_data.1136f033099c.0000000000000003:head v 146'664, mlcod=145'662)", 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:37.974597+0000", 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.001174388, 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001523763, 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974597+0000", 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974597+0000", 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974598+0000", 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3399999999999997e-07 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974607+0000", 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0270000000000001e-06 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974627+0000", 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0051000000000002e-05 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.974634+0000", 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0609999999999998e-06 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975123+0000", 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00048911400000000002 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975183+0000", 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0401e-05 2024-09-16T14:09:37.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:195342 3.16 3:6f4b5ad3:::rbd_data.1136f033099c.000000000000000a:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:37.975466+0000", 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.00030540299999999999, 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00073096499999999998, 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for sub ops", 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:37.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 195342 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975466+0000", 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:37.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975466+0000", 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975467+0000", 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4649999999999999e-06 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975472+0000", 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0220000000000004e-06 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975487+0000", 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4348999999999999e-05 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975492+0000", 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6319999999999996e-06 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975544+0000", 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2000999999999999e-05 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:37.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975648+0000", 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103422 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:37.975847+0000", 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019944400000000001 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.000 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:38.001 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 5 2024-09-16T14:09:38.001 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:38.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:38.045 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:38.041+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:38.100 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:38.101 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:38.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:38.146 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:38.145+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:38.157 DEBUG:teuthology.orchestra.run.smithi079:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op apply-layout-settings --pool pool_client.0 2024-09-16T14:09:38.213 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:38.213 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:38.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:09:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 66.687167703, 2024-09-16T14:09:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:09:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:09:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:09:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:09:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:09:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:09:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:09:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:09:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:09:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:09:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:09:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:09:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:09:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:09:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:09:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:09:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.681896363999996, 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:09:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:09:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.499387849999998, 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:09:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.495987319000001, 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:09:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:09:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:09:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:09:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.491936039999999, 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:09:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.488183124000003, 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:09:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.484103595999997, 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:09:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:09:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.484086994000002, 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:09:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:09:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.484071018000002, 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:09:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:09:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.484053066000001, 2024-09-16T14:09:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:09:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.479857934999998, 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.479760957000003, 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:09:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:09:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.479736416000001, 2024-09-16T14:09:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:09:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.476179653999999, 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:09:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:09:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.476161896000001, 2024-09-16T14:09:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:09:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.476144232999999, 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:09:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:09:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.476127038999998, 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:09:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:09:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.472430621000001, 2024-09-16T14:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:09:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:09:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:09:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.460100668999999, 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:09:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:09:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 23.786588497, 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:38.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:09:38.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:38.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:38.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:38.322 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:38.324 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:38.324 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:38.319+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:38.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:38.425 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:38.419+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:38.434 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:38.434 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:38.434 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:38.435 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:38.444 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:38.504 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:38.521 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:38.525 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:38.521+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:38.541 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:38.542 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:38.542 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:38.542 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:38.542 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:38.542 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:38.551 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:38.564 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:38.585 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:38.605 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:38.626 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:38.623+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:38.626 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:38.627 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:38.627 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:38.627 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:38.627 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:38.627 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:38.627 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:38.627 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:38.627 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:38.631 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:38.632 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:38.651 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:38.651 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:38.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:38.726 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:38.725+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:38.758 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:38.758 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:38.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:38.827 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:38.825+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:38.862 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:38.862 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:38.872 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:38.928 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:38.923+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:39.020 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.020 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.029 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:39.030 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:39.025+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:39.129 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.129 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.130 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:39.127+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:39.138 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:39.231 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:39.229+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:39.236 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.236 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.245 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:39.332 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:39.329+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:39.346 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.346 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.347 INFO:teuthology.orchestra.run.smithi079.stderr:Nothing to do for non-filestore backend 2024-09-16T14:09:39.356 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:39.433 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:39.431+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:39.502 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.502 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.511 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:39.534 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:39.529+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:39.635 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:39.631+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:39.713 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.713 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:39.735 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:39.733+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:39.835 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.835 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.836 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:39.831+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:39.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:39.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:09:39.937 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:39.931+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:39.963 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:39.963 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:39.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:40.038 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:40.037+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:40.129 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:40.129 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:40.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:40.140 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:40.137+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:40.236 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:40.236 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:40.241 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:40.235+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:40.245 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:40.342 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:40.342 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:40.342 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:40.337+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:40.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:40.443 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:40.441+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:40.449 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:40.449 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:40.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:40.544 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:40.539+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:40.558 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:40.559 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:40.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:40.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:40.644 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:40.641+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:40.667 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:40.668 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:40.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:40.745 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:40.741+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:40.767 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:40.770 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:40.770 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:40.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:40.786 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:40.802 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:40.823 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:40.845 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:40.846 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:40.841+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:40.873 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:40.892 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:40.893 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:40.893 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:40.893 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:40.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:40.897 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:40.897 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:40.897 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:40.898 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:40.898 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:40.898 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:40.898 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:40.898 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:40.902 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:40.903 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:40.903 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:09:40.904 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:40.947 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:40.945+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:40.997 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:40.997 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:41.006 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:41.047 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:41.043+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:41.121 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:41.121 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:41.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:41.121 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:41.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 143.778734345, 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:09:41.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:09:41.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:09:41.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:09:41.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 143.77871194799999, 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:09:41.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:09:41.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:09:41.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:09:41.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:41.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 143.77810523700001, 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:09:41.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:09:41.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 143.77803091000001, 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.142 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:41.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:09:41.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:09:41.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 133.80710611800001, 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:09:41.148 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:09:41.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:41.153 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 130.17914753700001, 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:41.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:09:41.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.159 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 129.64898707099999, 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.160 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.00143854, 2024-09-16T14:09:41.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:09:41.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.455031235999996, 2024-09-16T14:09:41.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:41.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:09:41.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.454997011000003, 2024-09-16T14:09:41.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:09:41.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:09:41.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:09:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:41.181 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.454967295000003, 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:09:41.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.454947633000003, 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.187 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:41.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:09:41.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:09:41.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:09:41.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.446187507999994, 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:09:41.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.197 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.361460965999996, 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.198 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:09:41.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:41.202 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.716457429000002, 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:09:41.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:09:41.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:09:41.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.209 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.716370568000002, 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.210 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.211 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:09:41.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.211 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:09:41.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.527063949000002, 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:09:41.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:09:41.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:09:41.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.641839726999997, 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:09:41.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:41.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:09:41.231 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 25.624776545, 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:09:41.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:09:41.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:09:41.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:09:41.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:09:41.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:09:41.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:194522 3.6 3:67cae2ed:::rbd_mirroring:head [call rbd.mirror_image_get in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.2283402580000002, 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02278196, 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T14:09:41.239 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 194522 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:41.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887296+0000", 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887310+0000", 2024-09-16T14:09:41.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3406e-05 2024-09-16T14:09:41.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909564+0000", 2024-09-16T14:09:41.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022254260000000001 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909597+0000", 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2869e-05 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909755+0000", 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158426 2024-09-16T14:09:41.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.910075+0000", 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031989900000000001 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.245 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:41.247 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:41.247 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:41.245+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:41.348 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:41.343+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:41.353 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:41.354 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:41.354 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:41.354 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:41.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:41.450 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:41.445+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:41.467 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:41.468 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:41.468 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:41.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:41.550 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:41.547+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:41.579 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:41.579 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:41.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:41.579 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.746643098999996, 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.580 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:41.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.746626759999998, 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:41.585 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:41.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:41.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.746601595000001, 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:41.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.592 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:41.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.745697628000002, 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:41.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:41.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.745688289, 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:41.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:41.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.745398167000005, 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.607 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:41.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:41.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:41.612 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.745179472999993, 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:41.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.617 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.738785894000003, 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.734874331, 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:41.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.734843126000001, 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:41.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.734023686, 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:41.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.640 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.733978876999998, 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:41.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:41.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:41.646 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.730969622999993, 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:41.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:41.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.730952662999997, 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.657 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.730941166999997, 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:41.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:41.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.726194257000003, 2024-09-16T14:09:41.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:41.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:41.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.718561625999996, 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:41.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:41.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:41.674 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.718549418999999, 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:41.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.716294504000004, 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:41.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 82.716070252999998, 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:41.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:41.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:41.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:41.691 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:41.693 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:41.694 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:41.689+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:41.793 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:41.789+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:41.797 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:41.797 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:41.798 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:41.798 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:41.807 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:41.894 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:41.893+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:41.957 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:41.957 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:41.957 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:41.966 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:42.015 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:42.013+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:42.074 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:42.074 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:42.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:42.074 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:42.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.074 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.074 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.694766673999993, 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:09:42.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.690835256, 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.079 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:09:42.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.690811064000002, 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.084 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:09:42.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.088 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.687091792999993, 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.089 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:09:42.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.683175027999994, 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:09:42.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.678792748999996, 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.099 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:09:42.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.678768907999995, 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:09:42.103 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:09:42.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.107 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.678753720000003, 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.108 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.112 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.678739133999997, 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:09:42.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:09:42.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.678635483999997, 2024-09-16T14:09:42.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.678607757999998, 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.122 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:09:42.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:09:42.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.675907428000002, 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.675867260999993, 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:09:42.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.675777546000006, 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:09:42.136 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:09:42.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:09:42.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.675268557999999, 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.141 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.667171187999998, 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:09:42.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:09:42.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.665002896000004, 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.151 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:42.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:09:42.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:42.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:09:42.156 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.515441435, 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:09:42.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [watch reconnect cookie 140383155395456 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.385432207000001, 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.013377462, 2024-09-16T14:09:42.163 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:09:42.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.694814+0000", 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003026674 2024-09-16T14:09:42.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.695138+0000", 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032383800000000001 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697164+0000", 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020260149999999999 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.379569072999999, 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.007999884, 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:09:42.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696135+0000", 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003762636 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696215+0000", 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0119999999999999e-05 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697528+0000", 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001312887 2024-09-16T14:09:42.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697582+0000", 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4409e-05 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697649+0000", 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7026000000000005e-05 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.176 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:42.177 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:42.178 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:42.171+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:42.278 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:42.273+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:42.284 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:42.284 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:42.284 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:42.293 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:42.379 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:42.375+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:42.441 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:42.441 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:42.441 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:42.442 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:42.442 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:42.450 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:42.480 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:42.477+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:42.552 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:42.552 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 141.13837059400001, 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:42.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:42.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:42.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 113.30040827099999, 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:42.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:42.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.773169283000001, 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.773146052000001, 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:42.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.773123104999996, 2024-09-16T14:09:42.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.580 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.773106596000005, 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:42.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.773088970000003, 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:42.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.767173225999997, 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:42.590 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.595 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.767155821000003, 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:42.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.600 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.766801455000007, 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:42.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.763657596000002, 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.605 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:42.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:42.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.763642309999994, 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.610 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:42.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.614 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.763625900999997, 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:42.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.763550703999996, 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:42.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.763453157000001, 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.624 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.763427915999998, 2024-09-16T14:09:42.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:42.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:42.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.633 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.763398862000003, 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:42.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:42.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.760134872999998, 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:42.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.760116671999995, 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.643 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:42.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:42.647 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.760077494000001, 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:42.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:42.652 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:42.652 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:42.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:42.654 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:42.647+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:42.755 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:42.753+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:42.761 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:42.761 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:42.761 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:42.771 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:42.855 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:42.853+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:42.868 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:42.868 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:42.869 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:42.869 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:42.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:42.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:42.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:09:42.956 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:42.953+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.745800661999994, 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:42.986 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:42.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:42.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385647618999997, 2024-09-16T14:09:42.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:42.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:42.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:42.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:42.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385638620000002, 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:42.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:42.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385630269000004, 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.003 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:43.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:43.008 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385597584999999, 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:43.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385588347999999, 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:43.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.018 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385579816000003, 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:43.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385571239000001, 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:43.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:43.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:43.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385561877999997, 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.029 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:43.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.033 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385553766000001, 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.034 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:43.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:43.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385542063000003, 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.039 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:43.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385533449999997, 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:43.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:43.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385525344000001, 2024-09-16T14:09:43.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:43.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:43.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385517002, 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:43.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:43.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385508463999997, 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:43.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:43.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385500131999997, 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.065 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:43.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.069 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385491889999997, 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:43.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:43.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:43.074 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385482326999998, 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.075 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:43.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:43.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385472946, 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:43.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:43.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.385463305000002, 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:43.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:43.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:43.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:43.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:43.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:43.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:43.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:43.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:43.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.092 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:43.092 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:43.092 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:43.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:43.095 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:43.095 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:43.095 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:43.095 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:43.087+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:43.126 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:43.154 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:43.175 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:43.195 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:43.193+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:43.202 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:43.202 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:43.202 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:43.202 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:43.202 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:43.202 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:43.203 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:43.203 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:43.203 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:43.204 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:43.204 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:43.205 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:43.206 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:43.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:43.296 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:43.291+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:43.306 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:43.307 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:43.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:43.396 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:43.391+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:43.413 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:43.413 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:43.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:43.498 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:43.493+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:43.525 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:43.526 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:43.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:43.599 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:43.595+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:43.635 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:43.635 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:43.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:43.700 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:43.695+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:43.786 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:43.786 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:43.796 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:43.843 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:43.841+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:43.902 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:43.897+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:43.902 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:43.903 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:43.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:43.934 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:09:44.002 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:44.001+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:44.027 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:44.103 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:44.099+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:44.175 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:44.175 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:44.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:44.204 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:44.201+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:44.280 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:44.280 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:44.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:44.331 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:44.329+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:44.405 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:44.401+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:44.433 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:44.433 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:44.442 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:44.506 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:44.501+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:44.541 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:44.541 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:44.550 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:44.607 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:44.605+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:44.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:44.707 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:44.705+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:44.808 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:44.808 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:44.808 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:44.803+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:44.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:44.905 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 1] out_osds: [7, 3] dead_osds: [0] live_osds: [1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:09:44.905 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:09:44.905 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T14:09:44.905 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-16T14:09:44.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T14:09:44.909 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:44.903+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:44.917 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:44.917 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:44.927 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:45.010 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:45.007+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:45.039 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:45.039 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:45.049 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:45.111 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:45.109+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:45.157 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:45.157 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:45.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:45.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:45.211 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:45.209+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:45.259 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:45.260 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:45.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:45.314 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:45.313+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:45.348 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:45.368 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:45.368 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:45.370 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:45.378 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:45.392 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:45.413 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:45.409+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:45.413 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:45.434 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:45.456 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:45.474 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:45.474 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:45.474 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:45.474 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:45.474 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:45.480 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:45.480 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:45.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:45.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:45.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:45.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:45.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:45.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:45.481 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:45.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:45.484 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:45.485 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:45.513 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:45.511+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:45.577 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.338307181000005, 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978154138000001, 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978145138999999, 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978136788, 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978104104000003, 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978094867000003, 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978086335, 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978077757999998, 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978068397000001, 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978060284999998, 2024-09-16T14:09:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978048582, 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978039969000001, 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978031862999998, 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978023520999997, 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978014983000001, 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.978006651000001, 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.977998409000001, 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.977988846000002, 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.977979464999997, 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.977969823999999, 2024-09-16T14:09:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:45.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:45.673 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:45.674 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:45.671+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:45.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:45.775 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:45.773+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:45.825 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:45.825 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:45.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:45.876 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:45.871+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:45.933 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:45.933 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:45.934 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:45.934 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:45.934 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:45.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:45.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:09:45.953 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T14:09:45.977 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:45.973+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:46.055 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:46.055 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:46.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:46.055 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:46.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.055 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.148390533, 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.14836782499999, 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.061 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:09:46.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:09:46.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.148343673, 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:09:46.067 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:09:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:46.073 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.148310732, 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:09:46.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.079 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.14828604900001, 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.080 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:46.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:09:46.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:09:46.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:09:46.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:09:46.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:09:46.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:09:46.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.14826282, 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:09:46.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.087 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:09:46.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:46.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:09:46.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:09:46.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.092 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.146416856, 2024-09-16T14:09:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:09:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.093 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:46.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:09:46.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.14637399199999, 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.099 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:09:46.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144914374, 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:09:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:09:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:09:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:09:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:09:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:09:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.14489860499999, 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:09:46.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144882853, 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:09:46.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144867422, 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:46.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144851784, 2024-09-16T14:09:46.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:46.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:09:46.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:09:46.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:09:46.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:09:46.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:09:46.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:09:46.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:09:46.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:09:46.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:09:46.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.135 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.136 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.136 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:46.136 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144836645, 2024-09-16T14:09:46.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:09:46.136 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.136 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:46.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:46.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:46.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:09:46.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:09:46.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:09:46.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:09:46.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:09:46.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:09:46.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:09:46.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:46.143 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144821808, 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:46.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:46.149 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144798804, 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:09:46.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:09:46.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.154 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.14478357900001, 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:09:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144768395, 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:09:46.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.167 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.14475315, 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:46.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:09:46.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:09:46.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:09:46.172 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.144737992, 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.173 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:46.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:09:46.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:09:46.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.178 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:46.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:46.180 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:46.177+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:46.281 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:46.281 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:46.281 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:46.282 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:46.282 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:46.275+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:46.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:46.382 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:46.377+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:46.385 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:46.385 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:46.386 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:46.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:46.483 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:46.481+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:46.493 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:46.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 74.971940228999998, 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:09:46.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:09:46.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:09:46.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.966668889999994, 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:09:46.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:09:46.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:09:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.506 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.784160376000003, 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.780759844999999, 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:09:46.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:09:46.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.516 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.776708566000003, 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.517 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:09:46.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.521 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.77295565, 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:46.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:46.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.768876122000002, 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.530 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.768859519999999, 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:09:46.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.768843543999999, 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:46.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.768825591999999, 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.540 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:09:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.764630461000003, 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:09:46.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.764533483000001, 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.549 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:46.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:09:46.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.764508941999999, 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:09:46.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.760952179999997, 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:09:46.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.760934421999998, 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:09:46.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:09:46.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.760916758999997, 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:09:46.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.760899565000003, 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:09:46.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.757203146999998, 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:46.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.744873194999997, 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.581 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:09:46.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:09:46.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 32.071361023000001, 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.586 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:09:46.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:46.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:09:46.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:46.592 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:46.593 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:46.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:46.595 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:46.589+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:46.695 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:46.691+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:46.700 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:46.700 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:46.700 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:46.709 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:46.796 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:46.793+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:46.815 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:46.815 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:46.815 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:46.815 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:46.815 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:46.824 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:46.897 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:46.893+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:46.928 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 149.585727421, 2024-09-16T14:09:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:09:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:09:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:09:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:09:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:09:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:09:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:09:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:09:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:09:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:09:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:09:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:09:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:09:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:09:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:09:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:09:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:09:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:09:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:09:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:09:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:09:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:09:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 149.58570502399999, 2024-09-16T14:09:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:09:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:09:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:09:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:09:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:09:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:09:46.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:46.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:09:46.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:09:46.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:09:46.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:09:46.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:46.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:09:46.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:09:46.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:46.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:09:46.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:09:46.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:46.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:09:46.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:09:46.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:46.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:09:46.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:09:46.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:46.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:09:46.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:09:46.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:46.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:09:46.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:09:46.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:46.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:09:46.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:09:46.960 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.960 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:46.960 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.960 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:46.961 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:46.961 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 149.585098313, 2024-09-16T14:09:46.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:09:46.961 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:46.961 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:46.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:46.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:46.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:09:46.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.963 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:46.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:46.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:46.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:46.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:46.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:46.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:09:46.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:09:46.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:46.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:09:46.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:09:46.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:46.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:09:46.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:09:46.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:09:46.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:09:46.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:46.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:09:46.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:09:46.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:46.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:09:46.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:09:46.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:46.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:09:46.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:09:46.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:46.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:09:46.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:09:46.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:46.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:09:46.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:09:46.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:46.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:09:46.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:09:46.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:46.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:09:46.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:09:46.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:46.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.976 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:46.976 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:46.976 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 149.58502398600001, 2024-09-16T14:09:46.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:09:46.977 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:46.977 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:46.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:46.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:46.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.978 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:09:46.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.978 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:46.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:46.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:46.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:46.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:46.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:46.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:09:46.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:09:46.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:46.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:09:46.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:09:46.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:46.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:09:46.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:09:46.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:09:46.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:09:46.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:46.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:09:46.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:09:46.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:46.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:09:46.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:09:46.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:46.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:09:46.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:09:46.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:09:46.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:46.988 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 139.614099194, 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:46.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:46.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:09:46.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:09:46.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:09:46.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:09:46.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.993 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.986140613, 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:09:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:46.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:09:46.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:46.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:46.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.45598014699999, 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:09:47.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:09:47.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:09:47.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.80843161600001, 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.006 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.010 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.26202431199999, 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:09:47.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.261990087, 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:09:47.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:09:47.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.261960371, 2024-09-16T14:09:47.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:09:47.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.261940709, 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.025 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:09:47.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:47.029 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 103.25318058400001, 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:09:47.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:09:47.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.168454041999993, 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.034 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.523450505, 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:09:47.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:09:47.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:09:47.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.523363644, 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.046 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:09:47.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:09:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:09:47.052 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.334057025, 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:09:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.448832803000002, 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:09:47.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:09:47.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.431769621000001, 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:09:47.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:09:47.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:09:47.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:194522 3.6 3:67cae2ed:::rbd_mirroring:head [call rbd.mirror_image_get in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.035333334000001, 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02278196, 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.071 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 194522 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887296+0000", 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887310+0000", 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3406e-05 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909564+0000", 2024-09-16T14:09:47.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022254260000000001 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909597+0000", 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2869e-05 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909755+0000", 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158426 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.910075+0000", 2024-09-16T14:09:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031989900000000001 2024-09-16T14:09:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.075 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:47.077 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:09:47.077 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:47.077 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:47.073+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:47.177 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:47.173+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:47.227 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:47.227 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:47.227 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:47.236 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:47.278 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:47.273+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:47.331 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:47.331 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:47.340 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:47.379 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:47.375+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:47.480 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:47.477+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:47.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.653438690000002, 2024-09-16T14:09:47.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:47.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:47.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:47.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.653422351000003, 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:47.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:47.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.499 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.653397186000007, 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:47.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.652493218999993, 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:47.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:47.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:47.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.652483880000005, 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:47.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:47.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:47.515 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.652193757999996, 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:47.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.651975063999998, 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:47.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.645581484999994, 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.526 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:47.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.641669922000005, 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.531 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:47.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.641638717000006, 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:47.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.640819277000006, 2024-09-16T14:09:47.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:47.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.546 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.640774468000004, 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:47.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.551 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.637765213999998, 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:47.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:47.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:47.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.637748254000002, 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:47.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.562 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.637736758000003, 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:47.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.567 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.632989847999994, 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.568 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:47.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.625357217000001, 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.573 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:47.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.625345010000004, 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:47.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:47.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:47.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.623090094999995, 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:47.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:47.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.622865844000003, 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:47.589 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:47.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:47.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:47.594 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:47.596 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:47.596 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:47.593+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:47.634 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:47.651 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:47.676 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:47.696 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:47.691+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:47.697 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:47.719 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:47.742 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:47.749 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:47.749 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:47.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:47.764 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:47.764 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:47.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:47.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:47.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:47.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:47.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:47.765 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:47.765 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:47.768 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:47.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:47.810 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:47.807+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:47.856 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:47.856 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:47.866 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:47.897 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:47.893+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:47.962 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:47.962 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:47.972 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:47.998 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:47.995+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:48.099 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:48.095+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:48.129 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:48.129 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:48.139 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:48.200 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:48.197+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:48.239 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:48.301 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:48.297+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:48.387 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:48.388 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:48.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:48.442 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:48.439+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:48.503 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:48.501+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:48.508 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:48.508 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:48.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:48.603 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:48.601+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:48.620 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:48.621 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:48.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:48.705 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:48.701+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:48.730 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:48.730 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:48.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:48.806 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:48.803+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:48.839 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:48.839 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:48.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:48.906 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:48.903+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:49.002 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.002 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.007 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:49.003+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:49.012 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:49.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:09:49.109 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:49.103+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:49.173 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.173 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:49.209 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:49.205+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:49.310 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:49.305+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:49.326 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.326 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:49.410 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:49.409+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:49.435 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.436 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:49.511 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:49.507+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:49.543 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.543 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.551 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:49.612 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:49.609+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:49.661 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.661 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.671 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:49.714 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:49.707+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:49.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:49.770 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.771 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:49.814 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:49.811+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:49.879 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.879 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.888 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:49.916 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:49.913+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:49.917 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:49.937 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:49.957 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:49.976 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:49.988 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:49.988 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:49.989 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:09:49.999 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:50.011 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:50.030 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:50.046 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:50.045+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:50.049 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:50.049 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:50.049 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:50.049 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:50.049 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:50.049 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:50.049 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:50.049 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:50.050 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:50.053 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:50.053 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:50.118 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:50.115+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:50.175 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:50.176 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:50.176 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:50.186 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:50.219 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:50.215+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:50.281 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:50.281 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.446375157999995, 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:50.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:50.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.286 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.446358818999997, 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.287 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:50.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.446333654, 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:50.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:50.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:50.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.445429687000001, 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:50.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:50.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.445420347999999, 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:50.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:50.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.445130226000003, 2024-09-16T14:09:50.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:50.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:50.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:50.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.444911532000006, 2024-09-16T14:09:50.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:50.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:50.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.438517953000002, 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:50.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:50.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:50.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:50.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.434606389999999, 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:50.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.329 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.434575185, 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:50.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:50.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:50.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.433755744999999, 2024-09-16T14:09:50.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:50.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:50.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.433710935999997, 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.341 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.430701682000006, 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:50.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.352 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.430684721999995, 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:50.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.430673225999996, 2024-09-16T14:09:50.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:50.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:50.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:50.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.366 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.425926316000002, 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:50.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:50.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.418293684999995, 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.372 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:50.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:50.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.418281477999997, 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:50.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:50.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.416026563000003, 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.389 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 91.415802311999997, 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:50.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.395 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:50.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:50.398 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:50.391+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:50.498 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:50.493+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:50.515 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:50.515 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:198158 3.1 3:87c4e2af:::rbd_object_map.113662e21c28:head [call lock.lock in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158)", 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:50.511360+0000", 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0030959899999999999, 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031296219999999999, 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 198158 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511360+0000", 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511360+0000", 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511362+0000", 2024-09-16T14:09:50.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0870000000000001e-06 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511372+0000", 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0509000000000001e-05 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511407+0000", 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4560000000000001e-05 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511420+0000", 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2782999999999999e-05 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511537+0000", 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117695 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511653+0000", 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011608300000000001 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.511881+0000", 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022730100000000001 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.513189+0000", 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013087050000000001 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514378+0000", 2024-09-16T14:09:50.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011884529999999999 2024-09-16T14:09:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514440+0000", 2024-09-16T14:09:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2300999999999999e-05 2024-09-16T14:09:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514489+0000", 2024-09-16T14:09:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9144999999999998e-05 2024-09-16T14:09:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(client.4406.0:198158 3.1 e158/23 ondisk, result = 0)", 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:50.514230+0000", 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00022567, 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00040035899999999998, 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514230+0000", 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514230+0000", 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514232+0000", 2024-09-16T14:09:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.623e-06 2024-09-16T14:09:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514241+0000", 2024-09-16T14:09:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1649999999999995e-06 2024-09-16T14:09:50.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514268+0000", 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7027999999999999e-05 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514277+0000", 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6889999999999999e-06 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514344+0000", 2024-09-16T14:09:50.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6317999999999998e-05 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514367+0000", 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2821999999999999e-05 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:50.514630+0000", 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026371400000000002 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.527 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:09:50.528 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 2 2024-09-16T14:09:50.528 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:50.529 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:50.599 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:50.595+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:50.624 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:50.624 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:50.624 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:50.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:50.700 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:50.697+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:50.731 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:50.731 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.352846517, 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:50.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:09:50.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:09:50.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.348915099, 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.737 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:09:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.741 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.348890907, 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:09:50.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:09:50.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.746 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.345171636, 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:09:50.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.751 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.341254871, 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:09:50.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:09:50.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.33687259200001, 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.757 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:50.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:09:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:09:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:09:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:09:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.761 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.33684875100001, 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.762 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:09:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:09:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:09:50.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.336833563, 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.33681897699999, 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:09:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.33671532699999, 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:09:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.33668760099999, 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:09:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:09:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:09:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:09:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:09:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:09:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:09:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:09:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:09:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:09:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:09:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:09:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:09:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:09:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:09:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.333987271, 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.793 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:09:50.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.797 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.333947104, 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:09:50.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.333857389, 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.802 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:09:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:09:50.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.33334840099999, 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:09:50.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:50.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.32525103099999, 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:50.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:09:50.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:09:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:09:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:09:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:09:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:09:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:09:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:09:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:09:50.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:09:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:09:50.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:50.820 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.323082739, 2024-09-16T14:09:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:09:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:09:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:09:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:09:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:09:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:09:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:09:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:09:50.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:09:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:09:50.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:09:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:09:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:09:50.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:09:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:09:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:09:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:09:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:09:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:09:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.832 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:09:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.173521278000003, 2024-09-16T14:09:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:09:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.833 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:09:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:09:50.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:09:50.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:09:50.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [watch reconnect cookie 140383155395456 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.043512049999997, 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.013377462, 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:09:50.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:50.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.694814+0000", 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003026674 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.695138+0000", 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032383800000000001 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697164+0000", 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020260149999999999 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.037648916000002, 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.007999884, 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:50.847 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:09:50.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:09:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:09:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696135+0000", 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003762636 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696215+0000", 2024-09-16T14:09:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0119999999999999e-05 2024-09-16T14:09:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:50.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697528+0000", 2024-09-16T14:09:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001312887 2024-09-16T14:09:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:50.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697582+0000", 2024-09-16T14:09:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4409e-05 2024-09-16T14:09:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697649+0000", 2024-09-16T14:09:50.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7026000000000005e-05 2024-09-16T14:09:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:50.856 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:50.856 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:50.858 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:50.858 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:50.853+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:50.958 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:50.955+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:50.963 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:50.963 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:50.964 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:50.964 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:50.966 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 1] out_osds: [7, 3] dead_osds: [0] live_osds: [1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:09:50.966 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:09:50.966 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T14:09:50.966 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T14:09:50.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:51.059 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:51.057+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:51.080 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:51.080 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:51.080 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:51.090 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:51.160 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:51.155+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:51.196 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:51.196 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:51.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:51.196 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:51.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 149.78226751299999, 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.197 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:51.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:51.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:51.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:51.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:51.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:51.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:51.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:51.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:51.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.94430518999999, 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.206 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:51.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:51.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:51.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:51.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:51.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:51.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:51.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:51.214 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.417066202000001, 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:51.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.417042971000001, 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:51.219 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:51.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.224 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.417020023999996, 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:51.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:51.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:51.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:51.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:51.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.417003515000005, 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.230 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:51.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:51.235 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.416985889000003, 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:51.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:51.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.411070144999997, 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.240 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:51.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.245 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.411052740000002, 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:51.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:51.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:51.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.410698374000006, 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:51.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:51.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.255 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.407554515000001, 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:51.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:51.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.407539228999994, 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:51.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:51.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.265 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.407522819999997, 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:51.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.407447622999996, 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.270 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:51.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.407350076, 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.276 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:51.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.276 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:51.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:51.281 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.407324834999997, 2024-09-16T14:09:51.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:51.281 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.281 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:51.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:51.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:51.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.407295781000002, 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:51.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.404031791999998, 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:51.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.404013590999995, 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:51.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.403974413, 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:51.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:51.305 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:51.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:51.308 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:51.305+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:09:51.308 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 159, 2024-09-16T14:09:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "a22b38a6-94db-470d-bf35-45d7605f3bb0", 2024-09-16T14:09:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2024-09-16T14:06:40.573726+0000", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2024-09-16T14:09:50.989867+0000", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2024-09-16T14:09:18.679093+0000", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2024-09-16T14:09:13.363709+0000", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2024-09-16T14:09:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 6, 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 3, 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "jewel", 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "quincy", 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2024-09-16T14:09:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:52.822826+0000", 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:09:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:09:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "18", 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:09:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:09:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:09:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:06:58.526935+0000", 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:09:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:09:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "22", 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 3, 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 22, 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:09:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:09:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:09:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:09:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:09:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:09:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:09:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:09:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:09:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:09:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:09:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:09:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:09:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:09:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:09:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:09:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:09:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:09:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:09:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:09:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:09:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-16T14:09:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "pool_client.0", 2024-09-16T14:09:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2024-09-16T14:07:08.375919+0000", 2024-09-16T14:09:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 8193, 2024-09-16T14:09:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T14:09:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2024-09-16T14:09:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2024-09-16T14:09:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2024-09-16T14:09:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2024-09-16T14:09:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2024-09-16T14:09:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2024-09-16T14:09:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T14:09:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T14:09:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2024-09-16T14:09:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2024-09-16T14:09:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 32, 2024-09-16T14:09:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 32, 2024-09-16T14:09:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 32, 2024-09-16T14:09:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 32, 2024-09-16T14:09:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 32, 2024-09-16T14:09:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2024-09-16T14:09:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2024-09-16T14:09:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2024-09-16T14:09:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2024-09-16T14:09:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2024-09-16T14:09:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2024-09-16T14:09:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2024-09-16T14:09:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "159", 2024-09-16T14:09:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2024-09-16T14:09:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T14:09:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T14:09:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2024-09-16T14:09:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2024-09-16T14:09:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 77, 2024-09-16T14:09:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 159, 2024-09-16T14:09:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2024-09-16T14:09:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2024-09-16T14:09:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2024-09-16T14:09:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2024-09-16T14:09:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2024-09-16T14:09:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2024-09-16T14:09:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2024-09-16T14:09:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2024-09-16T14:09:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2024-09-16T14:09:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2024-09-16T14:09:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2024-09-16T14:09:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T14:09:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T14:09:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T14:09:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2024-09-16T14:09:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2024-09-16T14:09:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2024-09-16T14:09:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2024-09-16T14:09:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2024-09-16T14:09:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2024-09-16T14:09:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2024-09-16T14:09:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2024-09-16T14:09:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2024-09-16T14:09:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2024-09-16T14:09:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T14:09:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2024-09-16T14:09:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2024-09-16T14:09:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2024-09-16T14:09:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2024-09-16T14:09:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2024-09-16T14:09:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2024-09-16T14:09:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2024-09-16T14:09:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2024-09-16T14:09:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2024-09-16T14:09:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:09:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a060b161-fd5a-4cfb-bac8-f2acc32d7955", 2024-09-16T14:09:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 0, 2024-09-16T14:09:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:09:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:09:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:09:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:09:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:09:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:09:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 128, 2024-09-16T14:09:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 130, 2024-09-16T14:09:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:09:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:09:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2024-09-16T14:09:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:09:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2024-09-16T14:09:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:09:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:09:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2024-09-16T14:09:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:09:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2024-09-16T14:09:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:09:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:09:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.344 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2024-09-16T14:09:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:09:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2024-09-16T14:09:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:09:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:09:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2024-09-16T14:09:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:09:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2024-09-16T14:09:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22196 2024-09-16T14:09:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6817/22196", 2024-09-16T14:09:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6819/22196", 2024-09-16T14:09:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6824/22196", 2024-09-16T14:09:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6821/22196", 2024-09-16T14:09:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:09:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "exists" 2024-09-16T14:09:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:09:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "f5ba4b39-ceb8-46b3-938e-57249ed84b87", 2024-09-16T14:09:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:09:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:09:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:09:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:09:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:09:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:09:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:09:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 130, 2024-09-16T14:09:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:09:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:09:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:09:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2024-09-16T14:09:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:09:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2024-09-16T14:09:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:09:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:09:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2024-09-16T14:09:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:09:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2024-09-16T14:09:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:09:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:09:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2024-09-16T14:09:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:09:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2024-09-16T14:09:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:09:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:09:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2024-09-16T14:09:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:09:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2024-09-16T14:09:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22195 2024-09-16T14:09:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6809/22195", 2024-09-16T14:09:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6811/22195", 2024-09-16T14:09:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6815/22195", 2024-09-16T14:09:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6813/22195", 2024-09-16T14:09:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:09:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:09:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:09:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:09:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "8b657f92-66c9-46a3-a31f-0bd0be414e4e", 2024-09-16T14:09:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:09:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:09:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:09:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:09:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:09:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:09:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:09:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 130, 2024-09-16T14:09:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:09:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:09:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:09:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2024-09-16T14:09:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:09:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2024-09-16T14:09:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:09:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:09:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2024-09-16T14:09:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:09:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2024-09-16T14:09:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:09:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:09:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2024-09-16T14:09:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:09:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2024-09-16T14:09:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:09:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:09:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22197 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6825/22197", 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6827/22197", 2024-09-16T14:09:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6831/22197", 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6829/22197", 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "15937bd7-8964-4140-9f86-b3093c318d1e", 2024-09-16T14:09:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 0, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 15, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 58, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 83, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 88, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 59, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6800", 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6801", 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:09:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2024-09-16T14:09:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 35927 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6801/35927", 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6803/35927", 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6807/35927", 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6805/35927", 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "714baf80-ade0-4e88-ad4e-5171add1fc6a", 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.94293212890625, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 14, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 130, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:09:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6808", 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6809", 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6810", 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6811", 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6814", 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:09:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6815", 2024-09-16T14:09:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:09:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6812", 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6813", 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22362 2024-09-16T14:09:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6809/22362", 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6811/22362", 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6815/22362", 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6813/22362", 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e5a21b84-81ab-45da-a081-3b504f5c4f3d", 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 128, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:09:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6800", 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6801", 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:09:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6802", 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6803", 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6806", 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6807", 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6804", 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6805", 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22363 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6801/22363", 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6803/22363", 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6807/22363", 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6805/22363", 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:09:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "a009857f-f798-4154-9ff4-f524f60b6c61", 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 130, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6824", 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6825", 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:09:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6826", 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6827", 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6830", 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6831", 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6828", 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6829", 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22364 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6825/22364", 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6827/22364", 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6831/22364", 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6829/22364", 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:09:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "e3506002-ba8b-4251-8ef1-136334574a6d", 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 0, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 34, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6816", 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6817", 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6818", 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6819", 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2024-09-16T14:09:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6822", 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6823", 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2024-09-16T14:09:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6820", 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.164:6821", 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 22365 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.164:6817/22365", 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.164:6819/22365", 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.164:6823/22365", 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.164:6821/22365", 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:09:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.104631+0000", 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 130 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:09:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987403+0000", 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.018134+0000", 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 65536, 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.987645+0000", 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 59 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:09:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.721507+0000", 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:09:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:47.869796+0000", 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.206576+0000", 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138320759226367, 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 65536, 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2024-09-16T14:06:48.038098+0000", 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2024-09-16T14:09:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [ 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "snaps": [ 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "begin": 30, 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "length": 24 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [ 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 3, 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "snaps": [ 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "begin": 30, 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "length": 1 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2024-09-16T14:09:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2024-09-16T14:09:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2024-09-16T14:09:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.417 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:51.418 INFO:tasks.thrashosds.thrasher:j is {'epoch': 159, 'fsid': 'a22b38a6-94db-470d-bf35-45d7605f3bb0', 'created': '2024-09-16T14:06:40.573726+0000', 'modified': '2024-09-16T14:09:50.989867+0000', 'last_up_change': '2024-09-16T14:09:18.679093+0000', 'last_in_change': '2024-09-16T14:09:13.363709+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T14:06:52.822826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T14:06:58.526935+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T14:07:08.375919+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 77, 'snap_epoch': 159, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': 'a060b161-fd5a-4cfb-bac8-f2acc32d7955', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 128, 'down_at': 130, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 22196}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 22196}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6824', 'nonce': 22196}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 22196}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 22196}]}, 'public_addr': '172.21.15.79:6817/22196', 'cluster_addr': '172.21.15.79:6819/22196', 'heartbeat_back_addr': '172.21.15.79:6824/22196', 'heartbeat_front_addr': '172.21.15.79:6821/22196', 'state': ['exists']}, {'osd': 1, 'uuid': 'f5ba4b39-ceb8-46b3-938e-57249ed84b87', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 130, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 22195}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 22195}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 22195}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 22195}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 22195}]}, 'public_addr': '172.21.15.79:6809/22195', 'cluster_addr': '172.21.15.79:6811/22195', 'heartbeat_back_addr': '172.21.15.79:6815/22195', 'heartbeat_front_addr': '172.21.15.79:6813/22195', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8b657f92-66c9-46a3-a31f-0bd0be414e4e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 130, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6823', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 22197}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 22197}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 22197}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 22197}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 22197}]}, 'public_addr': '172.21.15.79:6825/22197', 'cluster_addr': '172.21.15.79:6827/22197', 'heartbeat_back_addr': '172.21.15.79:6831/22197', 'heartbeat_front_addr': '172.21.15.79:6829/22197', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '15937bd7-8964-4140-9f86-b3093c318d1e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 58, 'up_from': 83, 'up_thru': 88, 'down_at': 59, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6800', 'nonce': 35927}, {'type': 'v1', 'addr': '172.21.15.79:6801', 'nonce': 35927}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 35927}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 35927}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 35927}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 35927}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 35927}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 35927}]}, 'public_addr': '172.21.15.79:6801/35927', 'cluster_addr': '172.21.15.79:6803/35927', 'heartbeat_back_addr': '172.21.15.79:6807/35927', 'heartbeat_front_addr': '172.21.15.79:6805/35927', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '714baf80-ade0-4e88-ad4e-5171add1fc6a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.94293212890625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 130, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 22362}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 22362}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 22362}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 22362}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 22362}]}, 'public_addr': '172.21.15.164:6809/22362', 'cluster_addr': '172.21.15.164:6811/22362', 'heartbeat_back_addr': '172.21.15.164:6815/22362', 'heartbeat_front_addr': '172.21.15.164:6813/22362', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e5a21b84-81ab-45da-a081-3b504f5c4f3d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 22363}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 22363}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 22363}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 22363}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 22363}]}, 'public_addr': '172.21.15.164:6801/22363', 'cluster_addr': '172.21.15.164:6803/22363', 'heartbeat_back_addr': '172.21.15.164:6807/22363', 'heartbeat_front_addr': '172.21.15.164:6805/22363', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a009857f-f798-4154-9ff4-f524f60b6c61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 130, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 22364}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 22364}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 22364}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 22364}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 22364}]}, 'public_addr': '172.21.15.164:6825/22364', 'cluster_addr': '172.21.15.164:6827/22364', 'heartbeat_back_addr': '172.21.15.164:6831/22364', 'heartbeat_front_addr': '172.21.15.164:6829/22364', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'e3506002-ba8b-4251-8ef1-136334574a6d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 34, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 22365}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 22365}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 22365}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 22365}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 22365}]}, 'public_addr': '172.21.15.164:6817/22365', 'cluster_addr': '172.21.15.164:6819/22365', 'heartbeat_back_addr': '172.21.15.164:6823/22365', 'heartbeat_front_addr': '172.21.15.164:6821/22365', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.104631+0000', 'dead_epoch': 130}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987403+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.018134+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.987645+0000', 'dead_epoch': 59}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.721507+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:47.869796+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.206576+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T14:06:48.038098+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 24}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T14:09:51.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:09:51.419 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:51.415+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:51.520 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:51.520 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:51.521 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:51.517+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:51.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:51.621 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:51.615+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:51.622 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:51.622 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:51.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:09:51.716 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:09:51.722 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:51.717+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:51.727 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-16T14:09:51.000343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1087105604,"num_objects":2980,"num_object_clones":597,"num_object_copies":5960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2980,"num_whiteouts":1072,"num_read":269693,"num_read_kb":908233,"num_write":28847,"num_write_kb":442976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5655,"num_bytes_recovered":288730627,"num_keys_recovered":802,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23518,"ondisk_log_size":23518,"up":68,"acting":68,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1882008,"kb_used_data":60404,"kb_used_omap":9,"kb_used_meta":1821494,"kb_avail":564349032,"statfs":{"total":579820584960,"available":577893408768,"internally_reserved":0,"allocated":61853696,"data_stored":486239361,"data_compressed":11087056,"data_compressed_allocated":37638144,"data_compressed_original":473313043,"omap_allocated":10224,"internal_metadata":1865209872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12670501,"num_objects":-88,"num_object_clones":-4,"num_object_copies":-176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-88,"num_whiteouts":-4,"num_read":556,"num_read_kb":463,"num_write":128,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035922"},"pg_stats":[{"pgid":"3.1f","version":"154'1733","reported_seq":7243,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:45.962249+0000","last_change":"2024-09-16T14:09:45.962552+0000","last_active":"2024-09-16T14:09:45.962249+0000","last_peered":"2024-09-16T14:09:45.962249+0000","last_clean":"2024-09-16T14:09:45.962249+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:09:45.962249+0000","last_undegraded":"2024-09-16T14:09:45.962249+0000","last_fullsized":"2024-09-16T14:09:45.962249+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1733,"log_dups_size":0,"ondisk_log_size":1733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:02.959438+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0087240359999999992,"stat_sum":{"num_bytes":17016636,"num_objects":80,"num_object_clones":11,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":23,"num_read":14631,"num_read_kb":18954,"num_write":2857,"num_write_kb":10262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.1e","version":"152'693","reported_seq":2424,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:45.953066+0000","last_change":"2024-09-16T14:09:45.953474+0000","last_active":"2024-09-16T14:09:45.953066+0000","last_peered":"2024-09-16T14:09:45.953066+0000","last_clean":"2024-09-16T14:09:45.953066+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:09:45.953066+0000","last_undegraded":"2024-09-16T14:09:45.953066+0000","last_fullsized":"2024-09-16T14:09:45.953066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069161500000000002,"stat_sum":{"num_bytes":29075536,"num_objects":99,"num_object_clones":16,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":30,"num_read":3354,"num_read_kb":21345,"num_write":889,"num_write_kb":11604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.1d","version":"152'527","reported_seq":4689,"reported_epoch":153,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:44.945319+0000","last_change":"2024-09-16T14:09:19.690707+0000","last_active":"2024-09-16T14:09:44.945319+0000","last_peered":"2024-09-16T14:09:44.945319+0000","last_clean":"2024-09-16T14:09:17.886039+0000","last_became_active":"2024-09-16T14:09:19.690707+0000","last_became_peered":"2024-09-16T14:09:19.690707+0000","last_unstale":"2024-09-16T14:09:44.945319+0000","last_undegraded":"2024-09-16T14:09:19.689098+0000","last_fullsized":"2024-09-16T14:09:19.688889+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:06:09.961932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072576699999999999,"stat_sum":{"num_bytes":54159276,"num_objects":117,"num_object_clones":26,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":38,"num_read":6860,"num_read_kb":24190,"num_write":711,"num_write_kb":15701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":560,"num_bytes_recovered":21960272,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":117}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"154'909","reported_seq":13763,"reported_epoch":154,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:46.135958+0000","last_change":"2024-09-16T14:09:19.691150+0000","last_active":"2024-09-16T14:09:46.135958+0000","last_peered":"2024-09-16T14:09:46.135958+0000","last_clean":"2024-09-16T14:09:17.833292+0000","last_became_active":"2024-09-16T14:09:19.691150+0000","last_became_peered":"2024-09-16T14:09:19.691150+0000","last_unstale":"2024-09-16T14:09:46.135958+0000","last_undegraded":"2024-09-16T14:09:19.689221+0000","last_fullsized":"2024-09-16T14:09:19.688999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:34:15.942448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064309800000000004,"stat_sum":{"num_bytes":38411574,"num_objects":100,"num_object_clones":23,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":100,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":41,"num_read":15508,"num_read_kb":26611,"num_write":1259,"num_write_kb":11025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":6143753,"num_keys_recovered":184,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":100}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"158'632","reported_seq":7352,"reported_epoch":158,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:50.459619+0000","last_change":"2024-09-16T14:09:19.694017+0000","last_active":"2024-09-16T14:09:50.459619+0000","last_peered":"2024-09-16T14:09:50.459619+0000","last_clean":"2024-09-16T14:09:17.689727+0000","last_became_active":"2024-09-16T14:09:19.694017+0000","last_became_peered":"2024-09-16T14:09:19.694017+0000","last_unstale":"2024-09-16T14:09:50.459619+0000","last_undegraded":"2024-09-16T14:09:19.693061+0000","last_fullsized":"2024-09-16T14:09:19.692940+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":23,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:07:51.860838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021275310000000002,"stat_sum":{"num_bytes":23332579,"num_objects":93,"num_object_clones":17,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":93,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":36,"num_read":9717,"num_read_kb":26018,"num_write":765,"num_write_kb":8170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":93}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"152'740","reported_seq":7328,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:46.142170+0000","last_change":"2024-09-16T14:09:45.946344+0000","last_active":"2024-09-16T14:09:46.142170+0000","last_peered":"2024-09-16T14:09:46.142170+0000","last_clean":"2024-09-16T14:09:46.142170+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:09:46.142170+0000","last_undegraded":"2024-09-16T14:09:46.142170+0000","last_fullsized":"2024-09-16T14:09:46.142170+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:25.215853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013950259999999999,"stat_sum":{"num_bytes":21430228,"num_objects":91,"num_object_clones":17,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":35,"num_read":9907,"num_read_kb":27994,"num_write":737,"num_write_kb":22826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.19","version":"158'567","reported_seq":2106,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-16T14:09:50.194810+0000","last_change":"2024-09-16T14:09:49.992270+0000","last_active":"2024-09-16T14:09:50.194810+0000","last_peered":"2024-09-16T14:09:50.194810+0000","last_clean":"2024-09-16T14:09:50.194810+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:09:50.194810+0000","last_undegraded":"2024-09-16T14:09:50.194810+0000","last_fullsized":"2024-09-16T14:09:50.194810+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:41.369426+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0094275699999999997,"stat_sum":{"num_bytes":27255081,"num_objects":65,"num_object_clones":11,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":1884,"num_read_kb":28639,"num_write":666,"num_write_kb":21949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"17"}]},{"pgid":"3.18","version":"154'856","reported_seq":5860,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:46.089173+0000","last_change":"2024-09-16T14:09:45.956002+0000","last_active":"2024-09-16T14:09:46.089173+0000","last_peered":"2024-09-16T14:09:46.089173+0000","last_clean":"2024-09-16T14:09:46.089173+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:09:46.089173+0000","last_undegraded":"2024-09-16T14:09:46.089173+0000","last_fullsized":"2024-09-16T14:09:46.089173+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028688979999999999,"stat_sum":{"num_bytes":35149971,"num_objects":76,"num_object_clones":11,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":26,"num_read":8706,"num_read_kb":21370,"num_write":1083,"num_write_kb":8662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.17","version":"154'1262","reported_seq":5785,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:45.956450+0000","last_change":"2024-09-16T14:09:45.956838+0000","last_active":"2024-09-16T14:09:45.956450+0000","last_peered":"2024-09-16T14:09:45.956450+0000","last_clean":"2024-09-16T14:09:45.956450+0000","last_became_active":"2024-09-16T14:09:14.390389+0000","last_became_peered":"2024-09-16T14:09:14.390389+0000","last_unstale":"2024-09-16T14:09:45.956450+0000","last_undegraded":"2024-09-16T14:09:45.956450+0000","last_fullsized":"2024-09-16T14:09:45.956450+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:11.631214+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0039724950000000004,"stat_sum":{"num_bytes":14090093,"num_objects":84,"num_object_clones":16,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":31,"num_read":8857,"num_read_kb":33223,"num_write":1929,"num_write_kb":5230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":4340034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.16","version":"154'613","reported_seq":13053,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:46.175420+0000","last_change":"2024-09-16T14:09:45.952059+0000","last_active":"2024-09-16T14:09:46.175420+0000","last_peered":"2024-09-16T14:09:46.175420+0000","last_clean":"2024-09-16T14:09:46.175420+0000","last_became_active":"2024-09-16T14:08:59.548051+0000","last_became_peered":"2024-09-16T14:08:59.548051+0000","last_unstale":"2024-09-16T14:09:46.175420+0000","last_undegraded":"2024-09-16T14:09:46.175420+0000","last_fullsized":"2024-09-16T14:09:46.175420+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:08.850228+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0053311390000000004,"stat_sum":{"num_bytes":25169320,"num_objects":106,"num_object_clones":19,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":40,"num_read":14114,"num_read_kb":49916,"num_write":597,"num_write_kb":12269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":11193866,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.15","version":"149'481","reported_seq":4825,"reported_epoch":153,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:44.958727+0000","last_change":"2024-09-16T14:09:19.691972+0000","last_active":"2024-09-16T14:09:44.958727+0000","last_peered":"2024-09-16T14:09:44.958727+0000","last_clean":"2024-09-16T14:09:06.148200+0000","last_became_active":"2024-09-16T14:09:19.691972+0000","last_became_peered":"2024-09-16T14:09:19.691972+0000","last_unstale":"2024-09-16T14:09:44.958727+0000","last_undegraded":"2024-09-16T14:09:19.690559+0000","last_fullsized":"2024-09-16T14:09:19.690349+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:32.026801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012153120000000001,"stat_sum":{"num_bytes":35895714,"num_objects":117,"num_object_clones":31,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":53,"num_read":4963,"num_read_kb":28560,"num_write":467,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":117}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"154'614","reported_seq":7356,"reported_epoch":154,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:46.077094+0000","last_change":"2024-09-16T14:09:19.693678+0000","last_active":"2024-09-16T14:09:46.077094+0000","last_peered":"2024-09-16T14:09:46.077094+0000","last_clean":"2024-09-16T14:09:17.844231+0000","last_became_active":"2024-09-16T14:09:19.693678+0000","last_became_peered":"2024-09-16T14:09:19.693678+0000","last_unstale":"2024-09-16T14:09:46.077094+0000","last_undegraded":"2024-09-16T14:09:19.691712+0000","last_fullsized":"2024-09-16T14:09:19.691508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:56.580163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009984429999999999,"stat_sum":{"num_bytes":48019806,"num_objects":97,"num_object_clones":22,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":97,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":31,"num_read":9011,"num_read_kb":38622,"num_write":622,"num_write_kb":10303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":7086932,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":97}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":153,"state":"active+undersized","last_fresh":"2024-09-16T14:09:44.958525+0000","last_change":"2024-09-16T14:09:19.690958+0000","last_active":"2024-09-16T14:09:44.958525+0000","last_peered":"2024-09-16T14:09:44.958525+0000","last_clean":"2024-09-16T14:06:59.453408+0000","last_became_active":"2024-09-16T14:09:19.690958+0000","last_became_peered":"2024-09-16T14:09:19.690958+0000","last_unstale":"2024-09-16T14:09:44.958525+0000","last_undegraded":"2024-09-16T14:09:44.958525+0000","last_fullsized":"2024-09-16T14:09:19.690218+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:04.192616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'751","reported_seq":5306,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-16T14:09:50.326100+0000","last_change":"2024-09-16T14:09:49.992626+0000","last_active":"2024-09-16T14:09:50.326100+0000","last_peered":"2024-09-16T14:09:50.326100+0000","last_clean":"2024-09-16T14:09:50.326100+0000","last_became_active":"2024-09-16T14:09:19.695118+0000","last_became_peered":"2024-09-16T14:09:19.695118+0000","last_unstale":"2024-09-16T14:09:50.326100+0000","last_undegraded":"2024-09-16T14:09:50.326100+0000","last_fullsized":"2024-09-16T14:09:50.326100+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:21.389187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.008251764,"stat_sum":{"num_bytes":29610631,"num_objects":87,"num_object_clones":16,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":27,"num_read":5164,"num_read_kb":25501,"num_write":831,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":15688379,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"17"}]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":157,"state":"active+undersized","last_fresh":"2024-09-16T14:09:48.979024+0000","last_change":"2024-09-16T14:09:19.696719+0000","last_active":"2024-09-16T14:09:48.979024+0000","last_peered":"2024-09-16T14:09:48.979024+0000","last_clean":"2024-09-16T14:09:17.689938+0000","last_became_active":"2024-09-16T14:09:19.696719+0000","last_became_peered":"2024-09-16T14:09:19.696719+0000","last_unstale":"2024-09-16T14:09:48.979024+0000","last_undegraded":"2024-09-16T14:09:48.979024+0000","last_fullsized":"2024-09-16T14:09:19.693368+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:14.962085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"158'865","reported_seq":7323,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-16T14:09:50.360164+0000","last_change":"2024-09-16T14:09:49.992920+0000","last_active":"2024-09-16T14:09:50.360164+0000","last_peered":"2024-09-16T14:09:50.360164+0000","last_clean":"2024-09-16T14:09:50.360164+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:09:50.360164+0000","last_undegraded":"2024-09-16T14:09:50.360164+0000","last_fullsized":"2024-09-16T14:09:50.360164+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:58.173875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0086934860000000003,"stat_sum":{"num_bytes":67690610,"num_objects":90,"num_object_clones":18,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":38,"num_read":9983,"num_read_kb":45304,"num_write":1181,"num_write_kb":30454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"17"}]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T14:09:48.978504+0000","last_change":"2024-09-16T14:09:02.367537+0000","last_active":"2024-09-16T14:09:48.978504+0000","last_peered":"2024-09-16T14:09:48.978504+0000","last_clean":"2024-09-16T14:09:48.978504+0000","last_became_active":"2024-09-16T14:09:02.367168+0000","last_became_peered":"2024-09-16T14:09:02.367168+0000","last_unstale":"2024-09-16T14:09:48.978504+0000","last_undegraded":"2024-09-16T14:09:48.978504+0000","last_fullsized":"2024-09-16T14:09:48.978504+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:03:05.340583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"152'527","reported_seq":3586,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:46.130604+0000","last_change":"2024-09-16T14:09:45.948843+0000","last_active":"2024-09-16T14:09:46.130604+0000","last_peered":"2024-09-16T14:09:46.130604+0000","last_clean":"2024-09-16T14:09:46.130604+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:09:46.130604+0000","last_undegraded":"2024-09-16T14:09:46.130604+0000","last_fullsized":"2024-09-16T14:09:46.130604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001164003,"stat_sum":{"num_bytes":11465366,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":33,"num_read":4343,"num_read_kb":15952,"num_write":532,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"2.4","version":"0'0","reported_seq":147,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T14:09:44.944338+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:09:44.944338+0000","last_peered":"2024-09-16T14:09:44.944338+0000","last_clean":"2024-09-16T14:09:44.944338+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:09:44.944338+0000","last_undegraded":"2024-09-16T14:09:44.944338+0000","last_fullsized":"2024-09-16T14:09:44.944338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"153'688","reported_seq":9515,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:45.947035+0000","last_change":"2024-09-16T14:09:45.947892+0000","last_active":"2024-09-16T14:09:45.947035+0000","last_peered":"2024-09-16T14:09:45.947035+0000","last_clean":"2024-09-16T14:09:45.947035+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:09:45.947035+0000","last_undegraded":"2024-09-16T14:09:45.947035+0000","last_fullsized":"2024-09-16T14:09:45.947035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012268820000000001,"stat_sum":{"num_bytes":24098181,"num_objects":90,"num_object_clones":18,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":27,"num_read":10969,"num_read_kb":12576,"num_write":800,"num_write_kb":5344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.3","version":"154'1131","reported_seq":4171,"reported_epoch":154,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:46.101074+0000","last_change":"2024-09-16T14:09:19.696514+0000","last_active":"2024-09-16T14:09:46.101074+0000","last_peered":"2024-09-16T14:09:46.101074+0000","last_clean":"2024-09-16T14:09:10.282923+0000","last_became_active":"2024-09-16T14:09:19.696514+0000","last_became_peered":"2024-09-16T14:09:19.696514+0000","last_unstale":"2024-09-16T14:09:46.101074+0000","last_undegraded":"2024-09-16T14:09:19.692445+0000","last_fullsized":"2024-09-16T14:09:19.692241+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:38:41.489816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014467779999999999,"stat_sum":{"num_bytes":35300742,"num_objects":103,"num_object_clones":19,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":39,"num_read":8422,"num_read_kb":18504,"num_write":1413,"num_write_kb":10020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":128,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T14:09:44.958697+0000","last_change":"2024-09-16T14:09:14.399885+0000","last_active":"2024-09-16T14:09:44.958697+0000","last_peered":"2024-09-16T14:09:44.958697+0000","last_clean":"2024-09-16T14:09:44.958697+0000","last_became_active":"2024-09-16T14:09:14.386881+0000","last_became_peered":"2024-09-16T14:09:14.386881+0000","last_unstale":"2024-09-16T14:09:44.958697+0000","last_undegraded":"2024-09-16T14:09:44.958697+0000","last_fullsized":"2024-09-16T14:09:44.958697+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:43.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"154'1198","reported_seq":5793,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:46.169535+0000","last_change":"2024-09-16T14:09:45.953765+0000","last_active":"2024-09-16T14:09:46.169535+0000","last_peered":"2024-09-16T14:09:46.169535+0000","last_clean":"2024-09-16T14:09:46.169535+0000","last_became_active":"2024-09-16T14:08:41.365767+0000","last_became_peered":"2024-09-16T14:08:41.365767+0000","last_unstale":"2024-09-16T14:09:46.169535+0000","last_undegraded":"2024-09-16T14:09:46.169535+0000","last_fullsized":"2024-09-16T14:09:46.169535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1198,"log_dups_size":0,"ondisk_log_size":1198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:19:25.019511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010570919999999999,"stat_sum":{"num_bytes":48608322,"num_objects":114,"num_object_clones":24,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":40,"num_read":9414,"num_read_kb":24220,"num_write":1664,"num_write_kb":10073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":7223267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T14:09:44.943252+0000","last_change":"2024-09-16T14:08:38.421227+0000","last_active":"2024-09-16T14:09:44.943252+0000","last_peered":"2024-09-16T14:09:44.943252+0000","last_clean":"2024-09-16T14:09:44.943252+0000","last_became_active":"2024-09-16T14:08:38.420867+0000","last_became_peered":"2024-09-16T14:08:38.420867+0000","last_unstale":"2024-09-16T14:09:44.943252+0000","last_undegraded":"2024-09-16T14:09:44.943252+0000","last_fullsized":"2024-09-16T14:09:44.943252+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:52.563336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"152'692","reported_seq":6602,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:45.949109+0000","last_change":"2024-09-16T14:09:45.950080+0000","last_active":"2024-09-16T14:09:45.949109+0000","last_peered":"2024-09-16T14:09:45.949109+0000","last_clean":"2024-09-16T14:09:45.949109+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:09:45.949109+0000","last_undegraded":"2024-09-16T14:09:45.949109+0000","last_fullsized":"2024-09-16T14:09:45.949109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020986730000000001,"stat_sum":{"num_bytes":31530710,"num_objects":93,"num_object_clones":15,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":34,"num_read":7901,"num_read_kb":20874,"num_write":851,"num_write_kb":13634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"2.0","version":"0'0","reported_seq":87,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T14:09:44.943222+0000","last_change":"2024-09-16T14:08:56.498908+0000","last_active":"2024-09-16T14:09:44.943222+0000","last_peered":"2024-09-16T14:09:44.943222+0000","last_clean":"2024-09-16T14:09:44.943222+0000","last_became_active":"2024-09-16T14:08:56.498527+0000","last_became_peered":"2024-09-16T14:08:56.498527+0000","last_unstale":"2024-09-16T14:09:44.943222+0000","last_undegraded":"2024-09-16T14:09:44.943222+0000","last_fullsized":"2024-09-16T14:09:44.943222+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:33.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"158'596","reported_seq":14873,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-16T14:09:50.158167+0000","last_change":"2024-09-16T14:09:49.985208+0000","last_active":"2024-09-16T14:09:50.158167+0000","last_peered":"2024-09-16T14:09:50.158167+0000","last_clean":"2024-09-16T14:09:50.158167+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:09:50.158167+0000","last_undegraded":"2024-09-16T14:09:50.158167+0000","last_fullsized":"2024-09-16T14:09:50.158167+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:15.668078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002668246,"stat_sum":{"num_bytes":29343988,"num_objects":77,"num_object_clones":19,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":25,"num_read":14801,"num_read_kb":52697,"num_write":588,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"17"}]},{"pgid":"2.3","version":"20'1","reported_seq":95,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T14:09:44.952233+0000","last_change":"2024-09-16T14:08:56.507137+0000","last_active":"2024-09-16T14:09:44.952233+0000","last_peered":"2024-09-16T14:09:44.952233+0000","last_clean":"2024-09-16T14:09:44.952233+0000","last_became_active":"2024-09-16T14:08:56.504285+0000","last_became_peered":"2024-09-16T14:08:56.504285+0000","last_unstale":"2024-09-16T14:09:44.952233+0000","last_undegraded":"2024-09-16T14:09:44.952233+0000","last_fullsized":"2024-09-16T14:09:44.952233+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:50.448909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":43,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-16T14:09:44.944268+0000","last_change":"2024-09-16T14:09:14.399362+0000","last_active":"2024-09-16T14:09:44.944268+0000","last_peered":"2024-09-16T14:09:44.944268+0000","last_clean":"2024-09-16T14:09:44.944268+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:09:44.944268+0000","last_undegraded":"2024-09-16T14:09:44.944268+0000","last_fullsized":"2024-09-16T14:09:44.944268+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:18.538987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"158'621","reported_seq":3625,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-16T14:09:50.166125+0000","last_change":"2024-09-16T14:09:49.983942+0000","last_active":"2024-09-16T14:09:50.166125+0000","last_peered":"2024-09-16T14:09:50.166125+0000","last_clean":"2024-09-16T14:09:50.166125+0000","last_became_active":"2024-09-16T14:09:02.369639+0000","last_became_peered":"2024-09-16T14:09:02.369639+0000","last_unstale":"2024-09-16T14:09:50.166125+0000","last_undegraded":"2024-09-16T14:09:50.166125+0000","last_fullsized":"2024-09-16T14:09:50.166125+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:59.255383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001721354,"stat_sum":{"num_bytes":11762822,"num_objects":92,"num_object_clones":16,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":36,"num_read":4208,"num_read_kb":11713,"num_write":669,"num_write_kb":6035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2056974,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"17"}]},{"pgid":"3.9","version":"158'556","reported_seq":6319,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-16T14:09:50.187711+0000","last_change":"2024-09-16T14:09:49.993910+0000","last_active":"2024-09-16T14:09:50.187711+0000","last_peered":"2024-09-16T14:09:50.187711+0000","last_clean":"2024-09-16T14:09:50.187711+0000","last_became_active":"2024-09-16T14:08:59.547175+0000","last_became_peered":"2024-09-16T14:08:59.547175+0000","last_unstale":"2024-09-16T14:09:50.187711+0000","last_undegraded":"2024-09-16T14:09:50.187711+0000","last_fullsized":"2024-09-16T14:09:50.187711+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:06:25.443307+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011407888,"stat_sum":{"num_bytes":17266185,"num_objects":82,"num_object_clones":17,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":32,"num_read":6652,"num_read_kb":16602,"num_write":537,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":8690877,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"17"}]},{"pgid":"3.a","version":"154'672","reported_seq":6310,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:45.945323+0000","last_change":"2024-09-16T14:09:45.945323+0000","last_active":"2024-09-16T14:09:45.945323+0000","last_peered":"2024-09-16T14:09:45.945323+0000","last_clean":"2024-09-16T14:09:45.945323+0000","last_became_active":"2024-09-16T14:09:17.690633+0000","last_became_peered":"2024-09-16T14:09:17.690633+0000","last_unstale":"2024-09-16T14:09:45.945323+0000","last_undegraded":"2024-09-16T14:09:45.945323+0000","last_fullsized":"2024-09-16T14:09:45.945323+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:01.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001089062,"stat_sum":{"num_bytes":70508664,"num_objects":116,"num_object_clones":25,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":41,"num_read":6607,"num_read_kb":38581,"num_write":655,"num_write_kb":22951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":481,"num_bytes_recovered":48816457,"num_keys_recovered":173,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.b","version":"152'639","reported_seq":4491,"reported_epoch":153,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:44.958584+0000","last_change":"2024-09-16T14:09:19.693402+0000","last_active":"2024-09-16T14:09:44.958584+0000","last_peered":"2024-09-16T14:09:44.958584+0000","last_clean":"2024-09-16T14:09:18.430165+0000","last_became_active":"2024-09-16T14:09:19.693402+0000","last_became_peered":"2024-09-16T14:09:19.693402+0000","last_unstale":"2024-09-16T14:09:44.958584+0000","last_undegraded":"2024-09-16T14:09:19.690885+0000","last_fullsized":"2024-09-16T14:09:19.690689+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:57.702057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056925200000000002,"stat_sum":{"num_bytes":29221193,"num_objects":95,"num_object_clones":22,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":95,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":41,"num_read":6583,"num_read_kb":44231,"num_write":741,"num_write_kb":22790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":9737480,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":95}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"158'519","reported_seq":7177,"reported_epoch":158,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:50.518773+0000","last_change":"2024-09-16T14:09:19.693694+0000","last_active":"2024-09-16T14:09:50.518773+0000","last_peered":"2024-09-16T14:09:50.518773+0000","last_clean":"2024-09-16T14:09:10.368396+0000","last_became_active":"2024-09-16T14:09:19.693694+0000","last_became_peered":"2024-09-16T14:09:19.693694+0000","last_unstale":"2024-09-16T14:09:50.518773+0000","last_undegraded":"2024-09-16T14:09:19.692647+0000","last_fullsized":"2024-09-16T14:09:19.692532+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":23,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:01.360021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017881959999999999,"stat_sum":{"num_bytes":30801669,"num_objects":107,"num_object_clones":26,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":47,"num_read":7277,"num_read_kb":27604,"num_write":450,"num_write_kb":13032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":17732459,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":107}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"152'414","reported_seq":2441,"reported_epoch":153,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:44.945778+0000","last_change":"2024-09-16T14:09:19.694783+0000","last_active":"2024-09-16T14:09:44.945778+0000","last_peered":"2024-09-16T14:09:44.945778+0000","last_clean":"2024-09-16T14:09:17.685301+0000","last_became_active":"2024-09-16T14:09:19.694783+0000","last_became_peered":"2024-09-16T14:09:19.694783+0000","last_unstale":"2024-09-16T14:09:44.945778+0000","last_undegraded":"2024-09-16T14:09:19.689184+0000","last_fullsized":"2024-09-16T14:09:19.688985+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:47:55.024970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074836900000000003,"stat_sum":{"num_bytes":35247675,"num_objects":89,"num_object_clones":22,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":89,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":29,"num_read":8492,"num_read_kb":25169,"num_write":358,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":15696890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":89}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"153'414","reported_seq":6779,"reported_epoch":153,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:45.084870+0000","last_change":"2024-09-16T14:09:19.693975+0000","last_active":"2024-09-16T14:09:45.084870+0000","last_peered":"2024-09-16T14:09:45.084870+0000","last_clean":"2024-09-16T14:09:17.868794+0000","last_became_active":"2024-09-16T14:09:19.693975+0000","last_became_peered":"2024-09-16T14:09:19.693975+0000","last_unstale":"2024-09-16T14:09:45.084870+0000","last_undegraded":"2024-09-16T14:09:19.692802+0000","last_fullsized":"2024-09-16T14:09:19.692583+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:37.818828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013505660000000001,"stat_sum":{"num_bytes":36501630,"num_objects":87,"num_object_clones":20,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":87,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":32,"num_read":7486,"num_read_kb":31355,"num_write":431,"num_write_kb":10906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":87}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"152'841","reported_seq":6076,"reported_epoch":153,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:44.946464+0000","last_change":"2024-09-16T14:09:19.693190+0000","last_active":"2024-09-16T14:09:44.946464+0000","last_peered":"2024-09-16T14:09:44.946464+0000","last_clean":"2024-09-16T14:09:15.630100+0000","last_became_active":"2024-09-16T14:09:19.693190+0000","last_became_peered":"2024-09-16T14:09:19.693190+0000","last_unstale":"2024-09-16T14:09:44.946464+0000","last_undegraded":"2024-09-16T14:09:19.690125+0000","last_fullsized":"2024-09-16T14:09:19.689922+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:58.572032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372222,"stat_sum":{"num_bytes":49532318,"num_objects":97,"num_object_clones":17,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":97,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":27,"num_read":9010,"num_read_kb":23914,"num_write":1090,"num_write_kb":15632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":8217813,"num_keys_recovered":36,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":97}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"153'530","reported_seq":12261,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:46.107665+0000","last_change":"2024-09-16T14:09:45.945461+0000","last_active":"2024-09-16T14:09:46.107665+0000","last_peered":"2024-09-16T14:09:46.107665+0000","last_clean":"2024-09-16T14:09:46.107665+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:09:46.107665+0000","last_undegraded":"2024-09-16T14:09:46.107665+0000","last_fullsized":"2024-09-16T14:09:46.107665+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:30.500273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001121602,"stat_sum":{"num_bytes":28010312,"num_objects":83,"num_object_clones":16,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":25,"num_read":13034,"num_read_kb":23812,"num_write":690,"num_write_kb":16335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.11","version":"152'918","reported_seq":5962,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:45.946988+0000","last_change":"2024-09-16T14:09:45.947800+0000","last_active":"2024-09-16T14:09:45.946988+0000","last_peered":"2024-09-16T14:09:45.946988+0000","last_clean":"2024-09-16T14:09:45.946988+0000","last_became_active":"2024-09-16T14:09:19.696289+0000","last_became_peered":"2024-09-16T14:09:19.696289+0000","last_unstale":"2024-09-16T14:09:45.946988+0000","last_undegraded":"2024-09-16T14:09:45.946988+0000","last_fullsized":"2024-09-16T14:09:45.946988+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":918,"log_dups_size":0,"ondisk_log_size":918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:48:04.657639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001550996,"stat_sum":{"num_bytes":17743292,"num_objects":70,"num_object_clones":11,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":24,"num_read":6784,"num_read_kb":16620,"num_write":1253,"num_write_kb":11204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":7211407,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.12","version":"153'605","reported_seq":5735,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T14:09:45.947883+0000","last_change":"2024-09-16T14:09:45.949045+0000","last_active":"2024-09-16T14:09:45.947883+0000","last_peered":"2024-09-16T14:09:45.947883+0000","last_clean":"2024-09-16T14:09:45.947883+0000","last_became_active":"2024-09-16T14:09:00.363242+0000","last_became_peered":"2024-09-16T14:09:00.363242+0000","last_unstale":"2024-09-16T14:09:45.947883+0000","last_undegraded":"2024-09-16T14:09:45.947883+0000","last_fullsized":"2024-09-16T14:09:45.947883+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:56.705622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014398639999999999,"stat_sum":{"num_bytes":47676686,"num_objects":94,"num_object_clones":13,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":36,"num_read":6928,"num_read_kb":56023,"num_write":612,"num_write_kb":31067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":28258380,"num_keys_recovered":60,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"23","length":"13"}]},{"pgid":"3.13","version":"153'639","reported_seq":7110,"reported_epoch":153,"state":"active+undersized+degraded","last_fresh":"2024-09-16T14:09:45.082575+0000","last_change":"2024-09-16T14:09:19.693531+0000","last_active":"2024-09-16T14:09:45.082575+0000","last_peered":"2024-09-16T14:09:45.082575+0000","last_clean":"2024-09-16T14:09:10.000096+0000","last_became_active":"2024-09-16T14:09:19.693531+0000","last_became_peered":"2024-09-16T14:09:19.693531+0000","last_unstale":"2024-09-16T14:09:45.082575+0000","last_undegraded":"2024-09-16T14:09:19.691810+0000","last_fullsized":"2024-09-16T14:09:19.691645+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":19,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:38.340483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066482800000000001,"stat_sum":{"num_bytes":65588407,"num_objects":98,"num_object_clones":26,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":98,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":31,"num_read":8057,"num_read_kb":31484,"num_write":816,"num_write_kb":18383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":26839160,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":98}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1086515217,"num_objects":2976,"num_object_clones":597,"num_object_copies":5952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2976,"num_whiteouts":1072,"num_read":269627,"num_read_kb":908178,"num_write":28744,"num_write_kb":441346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5649,"num_bytes_recovered":288140202,"num_keys_recovered":802,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":52334592,"data_stored":487703506,"data_compressed":11322711,"data_compressed_allocated":38375424,"data_compressed_original":479588115,"omap_allocated":8948,"internal_metadata":0},"log_size":23443,"ondisk_log_size":23443,"up":52,"acting":52,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509479,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313576,"kb_used_data":7900,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94058264,"statfs":{"total":96636764160,"available":96315662336,"internally_reserved":0,"allocated":8089600,"data_stored":64416503,"data_compressed":1259647,"data_compressed_allocated":4726784,"data_compressed_original":62693259,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1850000000000001,"5min":1.498,"15min":1.498},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.52300000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.89500000000000002,"15min":0.92600000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.034,"5min":1.413,"15min":1.413},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.96499999999999997,"15min":0.998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.093,"5min":1.78,"15min":1.78},"last":0.98699999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.84399999999999997,"15min":0.85799999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1990000000000001,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:09:28 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.9740000000000002,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.79500000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.92900000000000005,"15min":0.96899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.149,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.94199999999999995,"15min":0.97199999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.337,"5min":2.391,"15min":2.391},"last":1.093}]},{"osd":5,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.94199999999999995,"15min":0.96399999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.872}]},{"osd":7,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.0329999999999999,"15min":1.0589999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.248,"5min":1.7,"15min":1.7},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.121,"15min":1.149},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.456,"5min":2.012,"15min":2.012},"last":0.96399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306744,"kb_used_data":9388,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94065096,"statfs":{"total":96636764160,"available":96322658304,"internally_reserved":0,"allocated":9613312,"data_stored":80197340,"data_compressed":1498244,"data_compressed_allocated":5783552,"data_compressed_original":78137630,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97299999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.78700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.79200000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.88500000000000001,"15min":0.89500000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.93999999999999995,"15min":0.96799999999999997},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.92000000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":0.96099999999999997,"15min":0.92900000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":10.999000000000001,"5min":10.999000000000001,"15min":10.999000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.081,"15min":1.0609999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":11.018000000000001,"5min":11.018000000000001,"15min":11.018000000000001},"last":0.79800000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:09:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.62},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.46000000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.92600000000000005,"15min":0.94899999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2250000000000001,"5min":1.601,"15min":1.601},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.97999999999999998,"15min":1},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.208,"5min":1.764,"15min":1.764},"last":0.879}]},{"osd":6,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.91700000000000004,"15min":0.94299999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3140000000000001,"5min":1.538,"15min":1.538},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":1.0329999999999999,"15min":1.069},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.2589999999999999,"5min":1.573,"15min":1.573},"last":0.85299999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.03,"15min":1.0529999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3029999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.093,"15min":1.131},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.4079999999999999,"5min":1.619,"15min":1.619},"last":0.98599999999999999}]}]},{"osd":3,"up_from":0,"seq":356482285585,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312212,"kb_used_data":10692,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94059628,"statfs":{"total":96636764160,"available":96317059072,"internally_reserved":0,"allocated":10948608,"data_stored":83905710,"data_compressed":1826120,"data_compressed_allocated":6340608,"data_compressed_original":81406252,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.495}]},{"osd":1,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.69899999999999995,"15min":0.71599999999999997},"min":{"1min":0.34799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0129999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.754,"15min":0.76500000000000001},"min":{"1min":0.42799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.99399999999999999,"5min":1.454,"15min":1.454},"last":0.61599999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:09:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.69099999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.84599999999999997,"15min":0.86199999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.58799999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.347,"15min":1.347},"last":0.89200000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.56899999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1559999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.94699999999999995,"15min":0.96599999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.004,"5min":1.948,"15min":1.948},"last":0.95699999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.91900000000000004,"15min":0.92900000000000005},"min":{"1min":0.65900000000000003,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.129,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.98899999999999999,"15min":1.008},"min":{"1min":0.68200000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2150000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.1439999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.98299999999999998,"15min":0.997},"min":{"1min":0.65000000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.21,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95199999999999996,"15min":0.95399999999999996},"min":{"1min":0.71499999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1040000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.012}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312728,"kb_used_data":11208,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94059112,"statfs":{"total":96636764160,"available":96316530688,"internally_reserved":0,"allocated":11476992,"data_stored":83336861,"data_compressed":1866620,"data_compressed_allocated":7127040,"data_compressed_original":81060138,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.80800000000000005,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.5890000000000004,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.80500000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.76000000000000001},"min":{"1min":0.39900000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.466,"5min":1.998,"15min":1.998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.004,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.40799999999999997}]},{"osd":3,"last update":"Mon Sep 16 14:09:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.42799999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.88100000000000001,"15min":0.90100000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.153,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1499999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.141,"15min":1.1379999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":7.6989999999999998,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.85699999999999998,"5min":1.0489999999999999,"15min":1.081},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.258,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.0149999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.125,"15min":1.113},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":7.6740000000000004,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.88700000000000001,"5min":1.075,"15min":1.107},"min":{"1min":0.58099999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3560000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.056}]},{"osd":7,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":1.0389999999999999,"15min":1.0720000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5700000000000001,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.171,"15min":1.157},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.7190000000000003,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":0.96399999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318876,"kb_used_data":13204,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94052964,"statfs":{"total":96636764160,"available":96310235136,"internally_reserved":0,"allocated":13520896,"data_stored":113642782,"data_compressed":3586280,"data_compressed_allocated":9162752,"data_compressed_original":111271618,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.67400000000000004,"15min":0.68899999999999995},"min":{"1min":0.36399999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66900000000000004,"15min":0.67000000000000004},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.84499999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.88500000000000001,"15min":0.90400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.5790000000000002,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.034,"5min":1.109,"15min":1.109},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.89800000000000002,"15min":0.91000000000000003},"min":{"1min":0.504,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.119},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.54500000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.401,"5min":2.234,"15min":2.234},"last":1.2809999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.91300000000000003,"15min":0.93300000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.196,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.153},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.96099999999999997,"15min":0.97299999999999998},"min":{"1min":0.48499999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.46}]},{"osd":6,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.1379999999999999,"15min":1.1679999999999999},"min":{"1min":0.59499999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4470000000000001,"5min":2.3839999999999999,"15min":2.3839999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.997,"15min":1.016},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.306,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.226}]},{"osd":7,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.018,"15min":1.0369999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.5009999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0169999999999999,"15min":1.0249999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.401,"5min":1.458,"15min":1.458},"last":1.343}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317872,"kb_used_data":8012,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94053968,"statfs":{"total":96636764160,"available":96311263232,"internally_reserved":0,"allocated":8204288,"data_stored":60740165,"data_compressed":1050145,"data_compressed_allocated":4497408,"data_compressed_original":58744146,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2809999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.36399999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":2.4630000000000001,"5min":1.248,"15min":1.046},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":37.805,"5min":37.805,"15min":37.805},"last":1.1040000000000001},{"interface":"front","average":{"1min":2.3300000000000001,"5min":1.2030000000000001,"15min":1.0149999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":37.857999999999997,"5min":37.857999999999997,"15min":37.857999999999997},"last":0.879}]},{"osd":2,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.88200000000000001,"15min":0.872},"min":{"1min":0.505,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.6479999999999997,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.872,"15min":0.86699999999999999},"min":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.97499999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:09:31 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.81999999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.95899999999999996,"15min":0.95499999999999996},"min":{"1min":0.60299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93799999999999994,"15min":0.94599999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.1859999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.97099999999999997,"15min":0.97699999999999998},"min":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.4079999999999999,"5min":1.429,"15min":1.429},"last":1.052},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.026,"15min":1.034},"min":{"1min":0.65800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4890000000000001,"5min":1.601,"15min":1.601},"last":1.165}]},{"osd":7,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.028,"15min":1.042},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4670000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.075},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0920000000000001,"15min":1.1020000000000001},"min":{"1min":0.78900000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.4410000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.129}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12013568,"data_stored":112933997,"data_compressed":3586280,"data_compressed_allocated":9162752,"data_compressed_original":111271618,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9588736,"data_stored":81870847,"data_compressed":1860413,"data_compressed_allocated":7090176,"data_compressed_original":80470314,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8986624,"data_stored":81928577,"data_compressed":1773223,"data_compressed_allocated":6225920,"data_compressed_original":80300332,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":987136,"data_stored":6801231,"data_compressed":235709,"data_compressed_allocated":716800,"data_compressed_original":6668288,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6311936,"data_stored":59839678,"data_compressed":1050145,"data_compressed_allocated":4497408,"data_compressed_original":58744146,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7712768,"data_stored":78731345,"data_compressed":1492853,"data_compressed_allocated":5746688,"data_compressed_original":77564190,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6242304,"data_stored":63540857,"data_compressed":1259647,"data_compressed_allocated":4726784,"data_compressed_original":62693259,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":2056974,"data_compressed":64441,"data_compressed_allocated":208896,"data_compressed_original":1875968,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:09:51.731 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [2, 0, 4, 7] 2024-09-16T14:09:51.731 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.5', '2', '0', '4', '7'] 2024-09-16T14:09:51.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.5 2 0 4 7 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.489257890000005, 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:09:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:09:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129104847000001, 2024-09-16T14:09:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:09:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:09:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:09:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:09:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:09:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:09:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129095847999999, 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:09:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:09:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:09:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129087497, 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129054813000003, 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:09:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:09:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129045576000003, 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:09:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:09:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:09:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:09:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:09:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:09:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:09:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:09:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129037044, 2024-09-16T14:09:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:09:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:09:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:09:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:09:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:09:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:09:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:09:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:09:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:09:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129028466999998, 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:09:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129019106000001, 2024-09-16T14:09:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:09:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:09:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:09:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:09:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.129010993999998, 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128999291, 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:09:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128990678000001, 2024-09-16T14:09:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:09:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:09:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:09:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:09:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:09:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:09:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:09:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:09:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:09:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:09:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:09:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:09:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:09:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:09:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:09:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:09:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128982571999998, 2024-09-16T14:09:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:09:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:09:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:09:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:09:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:09:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:09:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:09:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:09:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:09:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:09:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:09:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:09:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:09:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:09:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:09:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:09:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128974229999997, 2024-09-16T14:09:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:09:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:09:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:09:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:09:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:09:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128965692000001, 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:09:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128957360000001, 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:09:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:09:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:09:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:09:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:09:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:09:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:09:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128949118000001, 2024-09-16T14:09:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:09:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:09:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128939555000002, 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:09:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:09:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:09:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:09:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128930173999997, 2024-09-16T14:09:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:09:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 61.128920532999999, 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:09:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:51.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:51.851 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:51.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:51.853 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:51.847+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:51.953 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:51.949+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:51.996 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:52.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:09:52.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:52.054 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:52.049+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:52.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:52.112 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:52.112 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:52.113 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:52.113 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:52.113 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:52.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:52.154 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:52.151+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:52.199 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:52.230 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:52.249 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:52.255 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:52.251+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:52.261 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:52.262 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:52.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:52.276 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:52.298 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:52.322 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:52.344 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:52.344 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:52.345 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:52.345 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:52.345 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:52.345 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:52.345 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:52.345 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:52.345 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:52.347 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:52.347 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:52.355 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:52.353+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:52.374 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:52.374 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:52.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:52.457 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:52.453+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:52.528 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:52.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:52.558 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:52.555+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:52.635 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:52.635 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:52.644 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:52.686 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:52.683+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:52.755 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:52.755 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:52.760 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:52.757+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:52.763 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:52.861 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:52.857+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:52.866 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:52.866 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:52.875 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:52.962 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:52.959+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:52.976 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:52.976 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:52.985 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.5 pg_upmap_items mapping to [2->0,4->7] 2024-09-16T14:09:53.037 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:09:53.063 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:53.061+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:53.096 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:53.096 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:53.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:53.164 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:53.161+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:53.200 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:53.201 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:53.210 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:53.264 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:53.261+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:53.363 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:53.363 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:53.365 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:53.359+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:53.373 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:53.466 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:53.463+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:53.473 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:53.473 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:53.483 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:53.567 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:53.563+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:53.581 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:53.581 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:53.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:53.668 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:53.665+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:53.688 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:53.688 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:53.697 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:53.769 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:53.763+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:53.841 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:53.841 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:53.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:53.870 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:53.865+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:53.947 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:53.947 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:53.956 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:53.999 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:53.997+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:54.071 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:54.069+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:54.109 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:54.109 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:54.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:54.172 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:54.169+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:54.232 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:54.232 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:54.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:54.273 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:54.269+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:54.340 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:54.341 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:54.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:54.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:54.374 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:54.369+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:54.443 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:54.443 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:54.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:54.475 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:54.470+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:54.485 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:54.509 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:54.529 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:54.553 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:54.561 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:54.561 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:54.561 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:54.561 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:54.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:54.575 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:54.573+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:54.576 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:54.615 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:54.643 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:54.643 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:54.644 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:54.646 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:54.647 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:54.672 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:54.672 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:54.672 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:54.675 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:54.670+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:54.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:54.777 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:54.773+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:54.827 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.91919713599999, 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:09:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.91917442800001, 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:09:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.919150276, 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:09:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:09:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:09:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.919117335, 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:09:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:09:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:09:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:09:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.919092652, 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:09:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:09:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.919069423, 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:09:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:09:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.917223459, 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:09:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:09:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:09:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:09:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.917180595, 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:09:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.91572097700001, 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:09:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:09:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:09:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:09:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.91570520800001, 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:09:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:09:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:09:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.915689456, 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:09:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:09:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.915674025, 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:09:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:09:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:09:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.91565838699999, 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:09:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:09:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:09:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.91564324799999, 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:09:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.915628411, 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:09:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:09:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:09:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:09:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:09:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:09:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:09:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:09:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:09:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:09:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:09:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:09:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:09:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:09:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:09:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:09:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:09:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.915605407, 2024-09-16T14:09:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:09:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:09:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:09:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:09:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:09:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:09:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:09:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:09:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:09:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:09:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:09:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:09:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:09:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:09:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:09:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:09:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:09:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:09:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:09:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:09:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.915590182, 2024-09-16T14:09:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:09:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:09:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:09:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:09:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:09:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:09:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:09:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:09:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:09:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:09:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:09:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:09:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:09:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:09:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:09:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:09:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:09:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:09:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:09:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:09:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.915574998, 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:09:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:09:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:09:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:09:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:09:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:09:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:09:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:09:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:09:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.915559753, 2024-09-16T14:09:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:09:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.915544595, 2024-09-16T14:09:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:54.956 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:54.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:54.957 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:54.950+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:55.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:09:55.057 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:55.050+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:55.072 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:55.072 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:55.073 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:55.073 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:55.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:55.159 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:55.154+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:55.194 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:09:55.194 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:09:55.194 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:55.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:55.259 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:55.254+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:55.300 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:09:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:09:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:09:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 83.777302895999995, 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:09:55.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.772031557000005, 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:09:55.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:09:55.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.589523043, 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:09:55.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.586122512000003, 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:09:55.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.582071232999994, 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.323 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:09:55.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.578318316999997, 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.328 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:09:55.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.574238789000006, 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:09:55.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:09:55.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.574222187000004, 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:09:55.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:09:55.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.574206211000003, 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:09:55.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:55.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.574188258999996, 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:09:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:09:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:09:55.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:09:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:09:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:09:55.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:09:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:09:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:09:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:09:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:09:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:09:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:09:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:09:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:09:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:09:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.569993127999993, 2024-09-16T14:09:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:09:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:09:55.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:55.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:09:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:09:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:09:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:09:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:09:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:09:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:09:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:09:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:09:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:09:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:09:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:09:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:09:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.367 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:55.368 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.569896150000005, 2024-09-16T14:09:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:09:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:09:55.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.569871609000003, 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:09:55.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:09:55.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.566314847000001, 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:09:55.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.383 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.566297089000003, 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:09:55.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.566279425999994, 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:09:55.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:09:55.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:55.393 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.566262232, 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:09:55.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:55.398 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.562565813999996, 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:55.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:09:55.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:09:55.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.550235861999994, 2024-09-16T14:09:55.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:09:55.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.876723689999999, 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:09:55.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:09:55.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:09:55.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:09:55.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:09:55.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:09:55.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:09:55.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:09:55.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:09:55.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:09:55.416 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:55.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:55.418 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:55.413+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:55.520 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:55.514+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:55.526 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:55.526 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:55.526 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:55.526 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:55.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:55.622 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:55.617+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:55.635 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:55.635 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:55.635 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:55.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:55.636 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:55.644 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:55.723 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:55.721+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:55.745 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.4038022, 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:09:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.40377980299999, 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:09:55.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:09:55.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.403173092, 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:09:55.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:09:55.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:55.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.40309876500001, 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:55.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:09:55.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 148.432173973, 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:09:55.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:09:55.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:09:55.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 144.804215392, 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:09:55.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.780 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 144.27405492599999, 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:09:55.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:09:55.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.62650639500001, 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:09:55.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:09:55.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.08009909099999, 2024-09-16T14:09:55.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:09:55.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:09:55.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.080064866, 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.796 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.08003515, 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.800 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:09:55.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.080015488, 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:09:55.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:09:55.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 112.07125536300001, 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:09:55.809 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:55.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:55.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:09:55.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.986528820999993, 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.818 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:09:55.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.341525283999999, 2024-09-16T14:09:55.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:09:55.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:09:55.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.341438423, 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:09:55.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:09:55.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:09:55.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:09:55.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:09:55.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.152131804, 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:09:55.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.266907582000002, 2024-09-16T14:09:55.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:55.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:09:55.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:09:55.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:09:55.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:55.854 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.249844400000001, 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:09:55.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:09:55.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.861 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:194522 3.6 3:67cae2ed:::rbd_mirroring:head [call rbd.mirror_image_get in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.853408113, 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02278196, 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T14:09:55.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 194522 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887296+0000", 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:55.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887310+0000", 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3406e-05 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909564+0000", 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022254260000000001 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:55.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909597+0000", 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2869e-05 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909755+0000", 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158426 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.910075+0000", 2024-09-16T14:09:55.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031989900000000001 2024-09-16T14:09:55.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.867 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:55.867 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:55.867 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:55.869 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:55.869 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:55.865+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:55.970 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:55.966+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:56.018 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:56.018 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:56.018 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:56.018 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:56.027 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:09:56.070 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:56.065+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:56.133 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:56.134 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:56.134 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:56.134 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:56.134 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:56.143 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:56.171 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:56.166+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:56.245 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.409766380999997, 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:09:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.409750041999999, 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:09:56.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.256 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.409724877000002, 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:09:56.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:09:56.261 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.408820910000003, 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:09:56.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.267 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.408811571000001, 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:56.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.272 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.408521449000006, 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.273 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:09:56.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:09:56.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.408302754999994, 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.401909176000004, 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.283 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:09:56.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.288 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.397997613000001, 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:09:56.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.397966408000002, 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:09:56.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:09:56.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.299 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.397146968000001, 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:09:56.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.397102158999999, 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:09:56.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:09:56.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.394092904999994, 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:09:56.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.315 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.394075944999997, 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:09:56.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.394064448999998, 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:09:56.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:56.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.389317539000004, 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:56.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:09:56.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.381684907999997, 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.332 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:09:56.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.381672700999999, 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:09:56.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.379417786000005, 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:09:56.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:09:56.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.379193534999999, 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:09:56.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:09:56.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.352 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.353 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.353 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:56.354 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:56.355 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:56.350+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:56.455 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:56.450+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:56.461 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:56.461 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:56.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:56.556 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:56.553+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:56.570 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:56.570 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:56.570 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:56.570 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:56.579 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:56.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:09:56.681 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.302957689, 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:09:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.299026271, 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:09:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.299002079, 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:09:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:09:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.295282808, 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:09:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.291366043, 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:09:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:09:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.286983764, 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:09:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.286959923, 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:09:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.28694473500001, 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:09:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.286930149, 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:09:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:09:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.286826499, 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:09:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.286798773, 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:09:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.284098443, 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:09:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.284058276, 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:09:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:09:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.28396856099999, 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:09:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:09:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123986 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9994240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.283459573, 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070580215000000002, 2024-09-16T14:09:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123986 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393950+0000", 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394241+0000", 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000291422 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394245+0000", 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1480000000000002e-06 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394251+0000", 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3820000000000001e-06 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.394255+0000", 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.493e-06 2024-09-16T14:09:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464428+0000", 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070172240999999996 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464471+0000", 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464530+0000", 2024-09-16T14:09:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9354999999999997e-05 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.275362203, 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:09:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 108.27319391100001, 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.123632450000002, 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:09:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:09:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:09:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:09:56.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [watch reconnect cookie 140383155395456 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.993623221999997, 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.013377462, 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:09:56.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:09:56.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.694814+0000", 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003026674 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.695138+0000", 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032383800000000001 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697164+0000", 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020260149999999999 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.772 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.987760088000002, 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.007999884, 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:09:56.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:56.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:09:56.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696135+0000", 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003762636 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696215+0000", 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0119999999999999e-05 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697528+0000", 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001312887 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697582+0000", 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4409e-05 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697649+0000", 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7026000000000005e-05 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:56.779 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:56.780 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:56.690+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:56.780 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:56.781 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:56.777+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:56.881 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:56.874+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:56.895 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:56.895 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:56.895 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:56.905 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:56.982 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:56.976+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:57.004 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:57.004 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:57.004 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:57.004 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:57.004 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:57.005 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:57.013 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:57.028 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:57.051 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:57.079 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:57.082 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:57.080+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:57.103 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:57.121 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:57.142 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:57.142 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:57.142 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:57.142 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:57.142 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:57.142 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:57.142 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:57.142 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:57.143 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:57.145 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:57.146 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:57.160 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:57.160 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:57.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:57.214 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:57.210+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:57.283 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:57.283 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:57.284 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:57.278+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:57.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:09:57.385 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:57.378+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:57.392 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:57.392 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:57.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:09:57.486 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:57.482+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:57.551 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:57.551 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:57.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:09:57.587 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:57.584+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:57.660 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:57.661 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:57.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:09:57.688 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:09:57.684+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:09:57.772 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:57.772 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:57.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:09:57.789 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:57.782+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:57.882 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:57.883 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:57.890 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:57.886+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:57.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:09:57.991 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:57.986+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:58.036 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:58.036 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:58.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:09:58.049 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 1] out_osds: [7, 3] dead_osds: [0] live_osds: [1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:09:58.049 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:09:58.049 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-16T14:09:58.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-16T14:09:58.071 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:09:58.093 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:58.088+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:58.160 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:58.160 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:58.172 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:09:58.192 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:58.188+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:58.291 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:58.291 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:58.293 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:58.286+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:58.301 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:09:58.394 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:58.388+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:58.405 INFO:tasks.ceph.mon.a.smithi079.stderr:2024-09-16T14:09:58.398+0000 7f5ada529700 -1 mon.a@0(leader).osd e166 definitely_dead 0 2024-09-16T14:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:58.423 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:09:58.495 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:09:58.492+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:09:58.519 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:58.519 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:58.528 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:09:58.596 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:58.592+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:58.623 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:58.623 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:58.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:09:58.697 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:58.690+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:58.777 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:58.777 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:58.786 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:09:58.798 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:58.792+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:58.885 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:58.885 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:58.893 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:09:58.939 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:58.936+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:59.000 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:58.996+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:59.042 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:59.042 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:59.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:09:59.089 INFO:teuthology.orchestra.run.smithi079.stderr:marked in osd.3. 2024-09-16T14:09:59.094 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:09:59.101 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:09:59.096+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:09:59.103 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-16T14:09:59.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:09:59.162 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:59.162 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:59.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:09:59.202 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:59.198+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:59.268 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:59.269 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:59.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:09:59.300 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:09:59.302 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:59.300+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:59.319 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:09:59.337 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:09:59.358 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:09:59.374 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:09:59.375 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:09:59.378 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:09:59.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:09:59.401 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:09:59.403 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:59.398+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:59.424 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:09:59.424 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:09:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:09:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:09:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:09:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:09:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:09:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:09:59.425 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:09:59.428 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:09:59.429 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:09:59.504 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:09:59.498+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:09:59.529 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:59.529 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:59.529 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:09:59.529 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:59.538 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:09:59.605 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:59.598+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:59.634 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:59.634 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:09:59.634 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:09:59.634 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:09:59.634 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:59.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:09:59.706 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:09:59.702+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:09:59.743 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:09:59.743 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:09:59.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:09:59.744 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:09:59.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.744 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:09:59.744 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:59.744 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.32834343900001, 2024-09-16T14:09:59.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:09:59.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:59.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.745 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:09:59.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.746 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:59.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:09:59.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:09:59.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:09:59.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:09:59.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:09:59.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:09:59.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:09:59.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:09:59.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:09:59.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:09:59.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:09:59.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:09:59.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:09:59.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:09:59.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:09:59.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:09:59.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:09:59.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:09:59.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:09:59.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:09:59.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:59.758 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 130.49038111600001, 2024-09-16T14:09:59.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:09:59.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:59.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:09:59.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:09:59.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:09:59.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:09:59.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:09:59.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:09:59.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:09:59.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:09:59.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:09:59.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:09:59.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:09:59.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.963142128, 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:09:59.768 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:09:59.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:09:59.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:09:59.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:09:59.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:09:59.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:09:59.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:09:59.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:09:59.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:09:59.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.775 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.775 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.775 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:59.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.963118897, 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:59.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:09:59.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:59.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.96309595, 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:09:59.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:09:59.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.963079441, 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:09:59.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.789 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.963061815, 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:09:59.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.957146071, 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.794 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.957128666, 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:09:59.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.95677430000001, 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:09:59.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.953630441, 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:09:59.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.95361515499999, 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.812 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:09:59.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.813 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:59.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:09:59.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.953598746, 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:09:59.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:09:59.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.953523549, 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:09:59.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.826 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.953426002, 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:59.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:09:59.830 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.953400761, 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.831 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:09:59.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.953371707, 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.836 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.840 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.950107718, 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:09:59.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:09:59.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.95008951699999, 2024-09-16T14:09:59.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:09:59.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.849 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 100.950050339, 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:09:59.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:09:59.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:09:59.854 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:09:59.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:09:59.857 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:09:59.854+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:09:59.957 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:09:59.952+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:09:59.973 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:09:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:09:59.973 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:09:59.973 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:09:59.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:00.058 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:00.050+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:00.081 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:00.082 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:00.082 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:00.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:00.158 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:00.152+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:00.190 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:00.190 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:00.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:00.190 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:00.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.190 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:10:00.190 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:00.190 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 92.949671772000002, 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:00.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:10:00.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.195 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160913 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2732032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589518729000005, 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023601803000000001, 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.196 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160913 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596701+0000", 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8699999999999995e-07 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596702+0000", 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.113e-06 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596705+0000", 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0149999999999999e-06 2024-09-16T14:10:00.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596707+0000", 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8989999999999999e-06 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620210+0000", 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023503052 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620248+0000", 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7991999999999998e-05 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620302+0000", 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4245e-05 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.200 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589509730000003, 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:00.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:10:00.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589501378999998, 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:10:00.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.209 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589468694999994, 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:10:00.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589459457999993, 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:10:00.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589450925999998, 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:10:00.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589442348999995, 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.224 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:10:00.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:00.228 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589432987999999, 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:10:00.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589424875999995, 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.233 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:10:00.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589413172999997, 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:10:00.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:10:00.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.241 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589404560000006, 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.242 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:00.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589396453999996, 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.246 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:10:00.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589388111999995, 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:10:00.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:10:00.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589379574000006, 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:00.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.259 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589371241999999, 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589363000000006, 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:00.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.268 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589353437, 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:10:00.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:10:00.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.273 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589344056000002, 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:10:00.277 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.589334414999996, 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.278 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:10:00.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.282 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:00.284 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:00.278+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:00.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:00.384 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:00.380+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:00.432 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:00.432 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200434 3.11 3.2be482f1 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.33320214500000001, 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.33324945700000003, 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200434 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098242+0000", 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.39e-06 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098250+0000", 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0870000000000007e-06 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098270+0000", 2024-09-16T14:10:00.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9760999999999999e-05 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098281+0000", 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1258e-05 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.099140+0000", 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085896699999999998 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.099150+0000", 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0516999999999999e-05 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200437 3.a 3.981fc0ca (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.33298580999999999, 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.33314840600000001, 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:10:00.437 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200437 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098457+0000", 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9600000000000008e-07 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098462+0000", 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4070000000000001e-06 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098476+0000", 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4243999999999999e-05 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098486+0000", 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3950000000000007e-06 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098546+0000", 2024-09-16T14:10:00.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0717999999999997e-05 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098558+0000", 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1089e-05 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200439 3.a 3.747373ca (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.33294634200000001, 2024-09-16T14:10:00.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.33320681600000002, 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "waiting for peered", 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200439 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098496+0000", 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7000000000000004e-07 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098499+0000", 2024-09-16T14:10:00.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2710000000000001e-06 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098509+0000", 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0066e-05 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098515+0000", 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098588+0000", 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3070000000000003e-05 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098596+0000", 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9880000000000004e-06 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 3 2024-09-16T14:10:00.445 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:00.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:00.485 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:00.478+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:00.546 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:00.547 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:00.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:00.602 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:00.598+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:00.687 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:00.684+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:00.709 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44728 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3123200~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:10:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.80136587200001, 2024-09-16T14:10:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0180779769999999, 2024-09-16T14:10:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44728 2024-09-16T14:10:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:10:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903530+0000", 2024-09-16T14:10:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903531+0000", 2024-09-16T14:10:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:10:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903534+0000", 2024-09-16T14:10:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9129999999999999e-06 2024-09-16T14:10:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903542+0000", 2024-09-16T14:10:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4149999999999999e-06 2024-09-16T14:10:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903547+0000", 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3009999999999997e-06 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906788+0000", 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003240373 2024-09-16T14:10:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906792+0000", 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5210000000000004e-06 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921373+0000", 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014580987 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921451+0000", 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7476000000000002e-05 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921608+0000", 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015742100000000001 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44729 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3135488~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.801343164, 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018184354, 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44729 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903553+0000", 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.68e-07 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903556+0000", 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6699999999999998e-06 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903565+0000", 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903570+0000", 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906813+0000", 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032424630000000001 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906817+0000", 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5029999999999999e-06 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921628+0000", 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014811028 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921673+0000", 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4589000000000003e-05 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921737+0000", 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4120000000000003e-05 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.80131901199999, 2024-09-16T14:10:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:10:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:10:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:10:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:10:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.80128607100001, 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:10:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:10:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:10:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:10:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:10:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:10:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:10:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.801261388, 2024-09-16T14:10:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:10:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:10:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:10:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:10:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:10:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:10:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:10:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:10:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:10:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:10:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:10:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:10:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:10:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:10:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:10:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:10:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:10:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:10:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:10:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.80123815899999, 2024-09-16T14:10:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:10:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:10:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:10:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:10:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:10:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:10:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:10:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:10:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:10:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:10:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:10:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:10:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:10:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:10:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:10:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:10:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:10:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:10:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:10:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44969 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2939904~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:10:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.799392195, 2024-09-16T14:10:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018176272, 2024-09-16T14:10:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44969 2024-09-16T14:10:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:10:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905504+0000", 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905505+0000", 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.429e-06 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905513+0000", 2024-09-16T14:10:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000003e-06 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905529+0000", 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.609e-05 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905538+0000", 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7550000000000006e-06 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907233+0000", 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001694704 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907239+0000", 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923477+0000", 2024-09-16T14:10:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0162381279999999 2024-09-16T14:10:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923567+0000", 2024-09-16T14:10:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9574000000000007e-05 2024-09-16T14:10:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923680+0000", 2024-09-16T14:10:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000113176 2024-09-16T14:10:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.799349331, 2024-09-16T14:10:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:10:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:10:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:10:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:10:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:10:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:10:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:10:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:10:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:10:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:10:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:10:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:10:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:10:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:10:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:10:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:10:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:10:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:10:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:10:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.797889713, 2024-09-16T14:10:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:10:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:10:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:10:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.797873944, 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:10:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.79785819200001, 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:10:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:10:00.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.797842761, 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.797827123, 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:10:00.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:10:00.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.79781198400001, 2024-09-16T14:10:00.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:10:00.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.797797147, 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:10:00.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.797774143, 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:00.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:10:00.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:10:00.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.797758918, 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:10:00.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:10:00.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.79774373399999, 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:10:00.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:10:00.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.834 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.79772848899999, 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:10:00.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:10:00.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:10:00.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.797713331, 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:00.840 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:10:00.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:00.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:00.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:00.845 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:00.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:00.847 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:00.842+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:00.948 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:00.942+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:00.951 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:00.951 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:00.951 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:00.951 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:00.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:01.049 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:01.042+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:01.105 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:01.105 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:01.105 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:01.105 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:01.105 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:01.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:10:01.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:01.150 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:01.148+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:01.251 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:01.248+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:01.283 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:10:01.283 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.761216461000004, 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:10:01.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:10:01.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.755945122, 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:10:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:10:01.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:01.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.573436607999994, 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:10:01.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:10:01.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.570036076999997, 2024-09-16T14:10:01.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:10:01.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:10:01.304 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.565984798000002, 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.305 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:10:01.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.309 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.562231882000006, 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:10:01.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.558152354000001, 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:10:01.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.558135751999998, 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:10:01.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:10:01.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.558119775999998, 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:10:01.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.558101824000005, 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.328 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:10:01.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.553906693000002, 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:10:01.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.553809715, 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:10:01.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:10:01.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.553785173999998, 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:10:01.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.550228411999996, 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.346 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:10:01.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.350 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.550210653999997, 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:10:01.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.550192991000003, 2024-09-16T14:10:01.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:10:01.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:10:01.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.550175796999994, 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:10:01.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.364 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.546479379000004, 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:10:01.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.534149427000003, 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:01.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:10:01.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.860637255, 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:01.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:10:01.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:01.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:01.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:01.381 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:01.382 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:01.383 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:01.376+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:01.430 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:10:01.483 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:01.478+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:01.488 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:01.488 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:01.488 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:01.488 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:01.498 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:01.564 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:01.582 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:01.584 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:01.580+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:01.599 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:01.605 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:01.605 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:01.605 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:01.615 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:01.620 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:01.639 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:01.659 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:01.680 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:10:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:01.684 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:01.678+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:01.685 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:10:01.685 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:10:01.715 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:01.715 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:01.725 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:01.785 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:01.780+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:01.819 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:01.819 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:01.828 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:01.887 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:01.882+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:01.974 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:01.974 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:01.984 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:01.987 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:01.984+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:02.083 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:02.083 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:02.088 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:02.082+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:02.092 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:02.122 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:10:02.189 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:02.184+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:02.201 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:02.201 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:02.211 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:02.290 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:02.288+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:02.353 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:02.353 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:02.365 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:02.411 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:02.408+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:02.465 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:02.492 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:02.488+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:02.576 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:02.619 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:02.616+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:02.695 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:02.690+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:02.723 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:02.723 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:02.733 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:02.796 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:02.790+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:02.845 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:02.845 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:02.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:02.896 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:02.892+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:02.970 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:02.970 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:02.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:02.997 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:02.992+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:03.076 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:03.076 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:03.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:03.097 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:03.092+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:03.198 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:03.194+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:03.206 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:03.207 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:03.218 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:03.299 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:03.294+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:03.366 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:03.366 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:03.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:03.400 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:03.394+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:03.473 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:03.474 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:03.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:03.530 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:03.526+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:03.601 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:03.596+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:03.654 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:03.654 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:03.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:03.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:10:03.730 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:03.726+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:03.776 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:03.777 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:03.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:03.803 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:03.798+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:03.822 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:03.841 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:03.860 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:03.882 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:03.896 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:03.896 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:03.896 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:03.896 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:03.896 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:03.904 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:03.900+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:03.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:03.908 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:03.937 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:03.971 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:10:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:03.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:03.973 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:03.974 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:10:03.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:10:04.005 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:04.000+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:04.021 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 92.499355284000004, 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:10:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 82.494083945, 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:10:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:10:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:10:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:10:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:10:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:10:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:10:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:10:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.311575430999994, 2024-09-16T14:10:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:10:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:10:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:10:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:10:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:10:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:10:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:10:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:10:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:10:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:10:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:10:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:10:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:10:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:10:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:10:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:10:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.308174899999997, 2024-09-16T14:10:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:10:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:10:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:10:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:10:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:10:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:10:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:10:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:10:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:10:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:10:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:10:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:10:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:10:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:10:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:10:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:10:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.304123621000002, 2024-09-16T14:10:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:10:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:10:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:10:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:10:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:10:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:10:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:10:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:10:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:10:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:10:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:10:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:10:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:10:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:10:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:10:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:10:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.300370705000006, 2024-09-16T14:10:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:10:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:10:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.296291177000001, 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:10:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:10:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:10:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:10:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.296274574999998, 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:10:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:10:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:10:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.296258598999998, 2024-09-16T14:10:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:10:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.296240647000005, 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:10:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.292045516000002, 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:10:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.291948538, 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:10:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.291923996999998, 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.288367234999996, 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:10:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.288349476999997, 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:10:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.288331814000003, 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.288314619999994, 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:10:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:10:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.284618202000004, 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:10:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.272288250000003, 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:10:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:10:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 49.598776078, 2024-09-16T14:10:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:10:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:10:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:10:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:04.138 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:04.138 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 1, 3] out_osds: [7] dead_osds: [0] live_osds: [1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:10:04.138 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:10:04.138 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T14:10:04.139 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T14:10:04.139 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T14:10:04.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-16T14:10:04.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:10:04.141 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:04.142 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:04.134+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:04.241 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:04.234+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:04.245 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:04.245 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:04.245 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:04.255 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:04.275 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:04.275 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T14:10:04.275 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:04.343 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:04.340+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:04.353 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:04.354 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:04.354 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:04.362 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:04.443 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:04.440+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:04.463 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:04.463 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:04.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:04.463 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:04.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.463 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 167.12174952399999, 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:10:04.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:10:04.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.469 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 167.12172712700001, 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:10:04.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:10:04.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:10:04.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:10:04.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:10:04.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:10:04.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.475 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.476 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.476 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:04.477 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:04.477 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 167.121120416, 2024-09-16T14:10:04.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:10:04.478 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.478 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:10:04.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:04.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:04.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:10:04.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:10:04.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:10:04.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:10:04.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:10:04.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:10:04.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:10:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:10:04.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 167.121046089, 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.486 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:04.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:10:04.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:10:04.491 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 157.150121297, 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.492 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 153.522162716, 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:10:04.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 152.99200225000001, 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:10:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:04.509 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.344453719, 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:10:04.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.798046415, 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.514 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.79801218999999, 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:10:04.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.79798247399999, 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.523 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:10:04.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.79796281199999, 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.528 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.532 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.789202687, 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.704476145, 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:10:04.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:10:04.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:10:04.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:10:04.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:10:04.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:10:04.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:10:04.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:10:04.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:10:04.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.059472607999993, 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:10:04.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:10:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 84.059385746999993, 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:10:04.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:10:04.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:10:04.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.870079128, 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:10:04.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:10:04.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:10:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:10:04.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 58.984854906000002, 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:10:04.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.967791724000001, 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:10:04.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:10:04.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:10:04.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:10:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:10:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:194522 3.6 3:67cae2ed:::rbd_mirroring:head [call rbd.mirror_image_get in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-16T14:10:04.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 30.571355437000001, 2024-09-16T14:10:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02278196, 2024-09-16T14:10:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T14:10:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.579 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 194522 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:04.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887296+0000", 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887310+0000", 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3406e-05 2024-09-16T14:10:04.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909564+0000", 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022254260000000001 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909597+0000", 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2869e-05 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909755+0000", 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158426 2024-09-16T14:10:04.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.910075+0000", 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031989900000000001 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.583 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:04.585 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:04.585 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:04.580+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:04.686 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:04.680+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:04.695 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:04.696 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:04.696 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:04.696 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:04.705 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:04.787 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:04.784+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:04.800 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:04.801 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:04.801 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:04.801 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:04.801 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:04.810 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:04.888 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:04.882+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:04.911 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.076836173, 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.912 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:04.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:04.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:10:04.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:10:04.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:10:04.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:10:04.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:10:04.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:10:04.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:10:04.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:10:04.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:10:04.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.07681983400001, 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:10:04.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:10:04.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:10:04.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:10:04.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:10:04.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:10:04.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:10:04.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:10:04.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.07679466899999, 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:10:04.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:10:04.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:10:04.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:10:04.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:10:04.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:10:04.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:10:04.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:10:04.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:10:04.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:10:04.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:10:04.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:10:04.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:10:04.936 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.936 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.936 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.937 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.937 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:04.937 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.075890702, 2024-09-16T14:10:04.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:10:04.937 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.938 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.938 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:10:04.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.939 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:04.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:04.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:04.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:10:04.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:10:04.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:10:04.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:10:04.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:10:04.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:10:04.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:10:04.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:10:04.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:10:04.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:10:04.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:10:04.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:10:04.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:10:04.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:10:04.946 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.947 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.947 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.947 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.947 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:04.947 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.07588136299999, 2024-09-16T14:10:04.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:10:04.948 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.948 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.948 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.948 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.948 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.948 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:10:04.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.948 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:04.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:04.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:10:04.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:10:04.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:10:04.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:10:04.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:10:04.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:10:04.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.955 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.075591241, 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.956 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:04.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:10:04.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.075372547, 2024-09-16T14:10:04.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:10:04.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:10:04.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.068978968, 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:10:04.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.971 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.06506740499999, 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.972 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:10:04.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.973 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:04.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:10:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:10:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:10:04.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:10:04.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:10:04.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.06503619999999, 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:04.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:10:04.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:10:04.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:10:04.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.985 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.986 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.986 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:04.986 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.06421675999999, 2024-09-16T14:10:04.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:10:04.986 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.987 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.987 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.987 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:10:04.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.988 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:04.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:04.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:10:04.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:10:04.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:10:04.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:10:04.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:10:04.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:10:04.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:04.994 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.06417195100001, 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:10:04.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:04.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:04.999 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.061162697, 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:05.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.004 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.061145737, 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:10:05.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:10:05.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.061134241, 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:10:05.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.014 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.056387331, 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:10:05.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.0487547, 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:10:05.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:10:05.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.04874249300001, 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.035 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.046487578, 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:10:05.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.04626332700001, 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:10:05.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:10:05.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.045 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.046 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.046 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:05.047 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:05.048 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:05.042+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:05.125 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:10:05.147 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:05.144+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:05.160 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:05.161 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:05.161 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:05.161 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:05.170 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:05.248 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:05.242+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:05.263 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:05.263 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:05.263 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:05.263 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:05.264 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:05.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:05.349 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:05.342+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.99233352, 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:10:05.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:10:05.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.98840210199999, 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:10:05.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:10:05.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:10:05.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:10:05.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:10:05.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:10:05.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.382 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.98837791, 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:10:05.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:10:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.984658639, 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.387 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:10:05.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:05.391 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.980741874, 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:05.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:10:05.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:10:05.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:10:05.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:10:05.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:10:05.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:10:05.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:10:05.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:10:05.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:10:05.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:10:05.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:10:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:10:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:10:05.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:10:05.399 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.399 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.399 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.399 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.400 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:05.400 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.97635959500001, 2024-09-16T14:10:05.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:10:05.400 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.400 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.400 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.401 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:10:05.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.401 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:05.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:05.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:05.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:10:05.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:10:05.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:10:05.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:10:05.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:10:05.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:10:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:10:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:10:05.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:10:05.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:10:05.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:10:05.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:10:05.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:10:05.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.409 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.409 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.410 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:05.410 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.976335754, 2024-09-16T14:10:05.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:10:05.410 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.410 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.410 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.411 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:10:05.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.411 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:05.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:05.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:05.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:10:05.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:10:05.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:10:05.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:10:05.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:10:05.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:10:05.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:10:05.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:10:05.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:10:05.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:10:05.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:10:05.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:10:05.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:10:05.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.420 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.420 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:05.420 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.976320566, 2024-09-16T14:10:05.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:05.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:05.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:05.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:10:05.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:10:05.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:10:05.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:10:05.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:10:05.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:10:05.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:10:05.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:10:05.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:10:05.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:10:05.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:10:05.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:10:05.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:10:05.427 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.427 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.427 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.428 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.428 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:05.428 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.97630598000001, 2024-09-16T14:10:05.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:10:05.428 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.429 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.429 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.429 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:10:05.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.429 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:05.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:05.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:10:05.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:10:05.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:10:05.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.97620233000001, 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:05.438 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.97617460399999, 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:10:05.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.442 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.973474274, 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.443 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:10:05.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:10:05.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.973434107, 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.447 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:10:05.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:05.451 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.973344392, 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:05.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.455 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.96473803400001, 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:10:05.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.460 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.962569742, 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:10:05.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:10:05.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.813008280999995, 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:10:05.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:10:05.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:10:05.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [watch reconnect cookie 140383155395456 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:10:05.472 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.682999053000003, 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.013377462, 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:10:05.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:05.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.694814+0000", 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003026674 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.695138+0000", 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032383800000000001 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697164+0000", 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020260149999999999 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.677135919000001, 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.007999884, 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.478 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:10:05.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:10:05.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696135+0000", 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003762636 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696215+0000", 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0119999999999999e-05 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697528+0000", 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001312887 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697582+0000", 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4409e-05 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697649+0000", 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7026000000000005e-05 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:201111 3.12 3:4da97782:::rbd_header.11369f5c332a:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e172)", 2024-09-16T14:10:05.484 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0579610509999999, 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81574065299999998, 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 201111 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308827+0000", 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8559999999999998e-06 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308839+0000", 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2170999999999999e-05 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308872+0000", 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3034999999999998e-05 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308888+0000", 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5158e-05 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308987+0000", 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8924000000000004e-05 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.309001+0000", 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4504e-05 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.123915+0000", 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.814914052 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.124090+0000", 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017510599999999999 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.124565+0000", 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047484699999999998 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.489 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:05.491 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:05.491 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:05.488+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:05.592 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:05.588+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:05.600 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:05.600 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:05.600 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:05.600 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:05.609 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:05.693 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:05.688+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:05.755 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:05.756 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:05.756 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:05.756 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:05.756 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:05.765 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:05.794 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:05.790+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:05.860 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:05.861 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:05.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:05.861 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:05.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.861 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:10:05.861 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:05.861 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 164.447896466, 2024-09-16T14:10:05.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:10:05.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.862 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:10:05.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.863 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:05.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:10:05.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:10:05.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:10:05.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:10:05.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:10:05.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:05.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:10:05.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:10:05.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:10:05.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:10:05.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:10:05.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:10:05.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.871 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.872 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:10:05.872 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:05.872 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 136.609934143, 2024-09-16T14:10:05.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:10:05.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:05.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:10:05.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:10:05.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:10:05.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:10:05.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:10:05.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:10:05.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:10:05.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:10:05.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:10:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:10:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:05.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:10:05.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:10:05.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:10:05.881 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.082695155, 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.882 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.883 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:10:05.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.883 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:05.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:05.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:10:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:10:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:10:05.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:10:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:10:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:10:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:10:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:10:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:10:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:10:05.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:10:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:10:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:10:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.082671924, 2024-09-16T14:10:05.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:10:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.892 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:10:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:10:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:10:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:10:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:10:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:10:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:10:05.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:10:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:10:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:10:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:10:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:10:05.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:10:05.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:10:05.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:10:05.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.900 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.901 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.901 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:05.901 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.08264897700001, 2024-09-16T14:10:05.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:10:05.901 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.901 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:05.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:05.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:10:05.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:10:05.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:10:05.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:10:05.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:10:05.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:10:05.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:10:05.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:10:05.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:10:05.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.908 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.082632468, 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:10:05.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.913 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.082614842, 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.914 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:10:05.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:10:05.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:10:05.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:10:05.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:10:05.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:10:05.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:10:05.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:05.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.07669909800001, 2024-09-16T14:10:05.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:10:05.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.922 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.922 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.922 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:10:05.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.922 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:05.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:05.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:05.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:10:05.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:10:05.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:10:05.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:10:05.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:10:05.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:10:05.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:10:05.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:10:05.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:10:05.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:10:05.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:10:05.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:10:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:10:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.931 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.931 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:05.931 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.076681693, 2024-09-16T14:10:05.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:10:05.932 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.932 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.932 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:10:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:10:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:10:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:10:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:10:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:10:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:10:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:10:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:10:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:10:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:10:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:10:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:10:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:10:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.076327327, 2024-09-16T14:10:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:10:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:10:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.948 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.073183468, 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.949 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:10:05.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.073168182, 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:10:05.953 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:10:05.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.07315177300001, 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.958 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:10:05.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.07307657600001, 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:10:05.962 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:10:05.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.072979029, 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:10:05.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:10:05.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.07295378800001, 2024-09-16T14:10:05.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:05.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:10:05.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.072924734, 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:10:05.976 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:10:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.977 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:05.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:10:05.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:10:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:10:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:10:05.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:10:05.982 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:05.983 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.06966074499999, 2024-09-16T14:10:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:10:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:10:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:10:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:10:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:10:05.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:10:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:10:05.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:10:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:10:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:10:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:10:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:10:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:10:05.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.069642544, 2024-09-16T14:10:05.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.995 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 107.069603366, 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:05.996 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:05.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:06.000 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:06.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:10:06.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:06.002 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:05.996+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:06.102 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:06.100+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:06.116 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:06.116 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:06.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:06.153 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:06.178 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:06.201 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:06.203 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:06.200+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:06.224 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:06.224 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:06.226 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:06.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:06.252 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:06.274 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:06.299 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:10:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:06.302 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:10:06.302 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:10:06.304 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:06.300+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:06.336 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:06.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:06.405 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:06.398+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:06.459 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:06.459 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:06.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:06.514 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:06.510+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:06.578 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:06.578 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:06.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:06.606 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:06.602+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:06.693 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:06.693 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:06.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:06.707 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:06.704+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:06.808 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:06.804+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:06.814 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:06.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:06.909 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:06.902+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:06.922 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:06.923 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:06.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:07.011 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:07.006+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:07.085 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:07.085 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:07.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:07.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:10:07.143 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:07.140+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:07.199 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:07.200 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:07.209 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:07.212 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:07.208+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:07.313 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:07.306+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:07.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:07.413 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:07.406+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:07.424 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:07.424 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:07.434 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:07.514 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:07.510+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:07.537 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:07.537 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:07.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:07.615 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:07.612+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:07.642 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:07.643 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:07.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:07.716 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:07.710+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:07.751 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:07.751 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:07.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:07.817 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:07.812+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:07.859 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:07.859 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:07.868 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:07.918 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:07.912+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:08.013 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:08.013 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:08.020 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:08.016+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:08.022 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:08.121 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:08.116+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:08.167 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:08.168 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:08.177 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:08.222 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:08.218+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:08.274 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:08.275 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:08.283 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:08.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:10:08.350 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:08.346+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:08.385 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:08.385 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:08.395 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:08.401 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:10:08.423 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:08.418+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:08.452 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:08.475 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:08.489 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:08.490 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:08.494 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:08.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:08.517 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:08.525 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:08.520+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:08.538 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:08.560 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:08.584 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:10:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:08.585 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:08.587 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:10:08.588 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:10:08.598 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:08.598 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:08.598 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:08.598 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:08.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:08.650 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:08.646+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:08.727 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:08.722+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:08.750 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:08.750 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:08.751 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:08.751 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:08.751 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:08.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:08.828 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:08.824+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 101.62047352800001, 2024-09-16T14:10:08.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:10:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.866 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260311486000006, 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:10:08.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260303135000001, 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:10:08.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:10:08.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.875 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260270450999997, 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:10:08.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:08.880 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260261213999996, 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:08.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:10:08.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260252682000001, 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:10:08.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260244104999998, 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:08.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:10:08.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260234744000002, 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:10:08.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:08.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260226631999998, 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:10:08.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:10:08.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260214929, 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:10:08.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260206315999994, 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:08.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:08.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:10:08.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:10:08.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260198209999999, 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.916 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:10:08.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260189867999998, 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:10:08.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260181329999995, 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:08.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:10:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:10:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260172998000002, 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.930 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:10:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260164755999995, 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.934 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:10:08.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:10:08.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260155193000003, 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:10:08.938 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:08.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:08.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:10:08.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:10:08.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260145812000005, 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.943 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:10:08.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 78.260136170999999, 2024-09-16T14:10:08.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.951 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:201147 3.1d 3:b85ce93f:::rbd_header.11367eb52311:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.6998338799999999, 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22849376599999999, 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 201147 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:08.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157189+0000", 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.227e-06 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157199+0000", 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9469999999999998e-06 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157219+0000", 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0035e-05 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157228+0000", 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9369999999999993e-06 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157310+0000", 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1459999999999996e-05 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157322+0000", 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2395000000000001e-05 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385317+0000", 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.227994851 2024-09-16T14:10:08.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385438+0000", 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121371 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385681+0000", 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024254300000000001 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.956 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:08.957 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:08.958 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:08.954+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:08.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:09.059 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:09.054+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:09.112 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:09.112 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:09.112 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:09.112 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:09.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:09.166 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:09.162+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:09.218 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:09.219 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:09.219 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:09.219 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:09.219 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:09.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:09.260 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:09.254+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:09.287 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 1, 3] out_osds: [7] dead_osds: [0] live_osds: [1, 2, 4, 5, 6, 7, 3] 2024-09-16T14:10:09.288 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:10:09.288 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-16T14:10:09.288 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T14:10:09.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T14:10:09.290 INFO:tasks.ceph.osd.0:Started 2024-09-16T14:10:09.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:10:09.337 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42985035800001, 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:10:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42981741700001, 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:10:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42979273399999, 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:10:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:10:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:10:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.429769505, 2024-09-16T14:10:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:10:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42788067699999, 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42642105900001, 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:10:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42640528999999, 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:10:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.426389538, 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:10:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42637410699999, 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:10:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42635846900001, 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:10:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:10:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42634333000001, 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:10:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:10:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:10:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.426328493, 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:10:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:10:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42630548899999, 2024-09-16T14:10:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:10:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42629026399999, 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:10:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:10:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.42627508000001, 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:10:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.426259835, 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:10:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.426244677, 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:10:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200434 3.11 3:8f4127d4:::rbd_data.113619b3ee73.0000000000000003:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2351879789999991, 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0847031039999999, 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200434 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098242+0000", 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.39e-06 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098250+0000", 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0870000000000007e-06 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098270+0000", 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9760999999999999e-05 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098281+0000", 2024-09-16T14:10:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1258e-05 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.099140+0000", 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085896699999999998 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.099150+0000", 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0516999999999999e-05 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180747+0000", 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0815962020000001 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180857+0000", 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011097199999999999 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,3", 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180980+0000", 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012213400000000001 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.181989+0000", 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001009338 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182283+0000", 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029419 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182697+0000", 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041376599999999998 2024-09-16T14:10:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182774+0000", 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6965000000000006e-05 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182942+0000", 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016855700000000001 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200437 3.a 3:5303f819:::rbd_data.113619b3ee73.0000000000000006:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2349716439999998, 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0809326290000001, 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200437 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098457+0000", 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9600000000000008e-07 2024-09-16T14:10:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098462+0000", 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4070000000000001e-06 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098476+0000", 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4243999999999999e-05 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098486+0000", 2024-09-16T14:10:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3950000000000007e-06 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098546+0000", 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0717999999999997e-05 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098558+0000", 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1089e-05 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173098+0000", 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0745404629999999 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173311+0000", 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021308399999999999 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173553+0000", 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000241964 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.177396+0000", 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00384337 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179118+0000", 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017211419999999999 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179185+0000", 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7290000000000001e-05 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179388+0000", 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020346700000000001 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200439 3.a 3:53cece2e:::rbd_data.113619b3ee73.0000000000000008:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.2349321759999992, 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0811117130000001, 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200439 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098496+0000", 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7000000000000004e-07 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098499+0000", 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2710000000000001e-06 2024-09-16T14:10:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098509+0000", 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0066e-05 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098515+0000", 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098588+0000", 2024-09-16T14:10:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3070000000000003e-05 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098596+0000", 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9880000000000004e-06 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174400+0000", 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0758035370000001 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174548+0000", 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014881900000000001 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174732+0000", 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018332799999999999 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.178747+0000", 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040152809999999999 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179446+0000", 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00069927300000000004 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179505+0000", 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8573999999999999e-05 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179607+0000", 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101951 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.447 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:09.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:09.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T14:10:09.449 INFO:teuthology.orchestra.run.smithi079.stderr:no valid command found; 10 closest matches: 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:0 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:1 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:2 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:abort 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:assert 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:config diff 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:config diff get 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:config get 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:config help [] 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:config set ... 2024-09-16T14:10:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:admin_socket: invalid command 2024-09-16T14:10:09.451 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-16T14:10:09.451 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:09.444+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:09.549 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:09.542+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:09.563 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:09.563 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:09.563 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:09.564 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:09.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:09.649 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:09.644+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:09.668 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:09.662+0000 7f0ddd714540 -1 Falling back to public interface 2024-09-16T14:10:09.718 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:09.718 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:09.719 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:09.719 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:09.719 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:09.727 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:09.750 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:09.748+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:09.829 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.306249346000001, 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.830 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:10:09.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.300978006999998, 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.836 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:10:09.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:10:09.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.118469493000006, 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:10:09.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:10:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.847 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.115068961999995, 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:10:09.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.111017683, 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:10:09.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.856 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.107264767000004, 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:10:09.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:10:09.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:09.861 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.103185238999998, 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.103168636999996, 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:10:09.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.870 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.103152660999996, 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:09.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:10:09.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:10:09.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.103134709000003, 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:10:09.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:10:09.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.098939578, 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:10:09.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:10:09.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.098842599999998, 2024-09-16T14:10:09.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:10:09.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:10:09.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.098818058999996, 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:10:09.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.095261296999993, 2024-09-16T14:10:09.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:10:09.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:10:09.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.095243538999995, 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:10:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.902 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.095225876000001, 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:10:09.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:10:09.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.095208682000006, 2024-09-16T14:10:09.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:10:09.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.091512264000002, 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.912 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 79.079182312, 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:10:09.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.405670139999998, 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:10:09.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:10:09.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:09.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:09.928 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:09.928 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:09.929 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:09.929 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:09.924+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:10.030 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:10.026+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:10.038 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:10.038 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:10.038 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:10.038 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:10.047 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:10.131 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:10.128+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:10.147 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:10.148 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:10.157 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:10.232 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:10.228+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:10.260 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:10.260 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:10.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 172.918634664, 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:10:10.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:10:10.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.266 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 172.91861226699999, 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.267 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:10:10.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:10:10.272 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 172.918005556, 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.273 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:10:10.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:10:10.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 172.917931229, 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.279 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:10:10.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:10:10.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 162.947006437, 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:10:10.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:10:10.289 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 159.319047856, 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:10.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:10:10.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:10:10.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 158.78888739000001, 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:10:10.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:10:10.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 131.141338859, 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:10:10.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:10:10.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:10.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.594931555, 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:10.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:10:10.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.59489732999999, 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.311 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:10:10.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.315 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.59486761399999, 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:10:10.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:10:10.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.59484795199999, 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.320 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:10.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.586087827, 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:10.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:10:10.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.501361285, 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:10:10.329 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:10:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.856357747999994, 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.334 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:10:10.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:10:10.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:10:10.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:10:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:10:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.856270886999994, 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:10:10.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:10:10.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:10:10.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:10:10.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.666964268000001, 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.349 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:10:10.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.781740045999996, 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:10:10.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:10:10.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:10:10.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.764676864000002, 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:10:10.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:10:10.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:10.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.367 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:194522 3.6 3:67cae2ed:::rbd_mirroring:head [call rbd.mirror_image_get in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.368240577000002, 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02278196, 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.368 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 194522 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887296+0000", 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887310+0000", 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3406e-05 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909564+0000", 2024-09-16T14:10:10.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022254260000000001 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909597+0000", 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2869e-05 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909755+0000", 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158426 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.910075+0000", 2024-09-16T14:10:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031989900000000001 2024-09-16T14:10:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.372 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.372 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:10.374 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:10.374 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:10.368+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:10.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:10:10.475 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:10.470+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:10.511 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:10.511 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:10.511 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:10.520 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:10.575 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:10.572+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:10.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:10:10.632 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:10.632 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:10.632 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:10.632 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:10.632 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:10.641 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:10.676 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:10.670+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.901977266, 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:10:10.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.742 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.901960927, 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:10:10.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:10:10.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.747 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.90193576199999, 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.748 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:10:10.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.753 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.90103179499999, 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:10:10.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.90102245600001, 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:10:10.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:10:10.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.900732334, 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.764 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:10.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:10:10.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.90051364, 2024-09-16T14:10:10.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:10:10.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.774 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.894120061, 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:10:10.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:10:10.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.89020849800001, 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:10.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:10:10.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:10.785 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.89017729299999, 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:10:10.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.790 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.88935785300001, 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:10:10.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.889313044, 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:10:10.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:10:10.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.88630379, 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:10:10.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:10:10.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.88628683, 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.807 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:10.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:10:10.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:10:10.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:10.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.886275334, 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:10.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:10:10.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.881528424, 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:10:10.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.822 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.873895793, 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.823 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:10:10.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.87388358600001, 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:10:10.828 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:10:10.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.833 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.871628671, 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:10:10.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 111.87140442, 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:10.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:10:10.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:10.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:10:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.844 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:10.844 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:10.844 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:10.845 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:10.845 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:10.845 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:10.846 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:10.846 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:10.838+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:10.846 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:10.852 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:10.874 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:10.899 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:10.899 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:10:10.899 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:10.899 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:10.899 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:10.899 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:10.900 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:10.900 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:10.900 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:10.903 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:10:10.903 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:10:10.946 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:10.942+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:10.957 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:10.957 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:10.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:11.047 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:11.042+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:11.059 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:11.059 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:11.068 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:11.148 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:11.144+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:11.164 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:11.165 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:11.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:11.248 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:11.242+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:11.267 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:11.267 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:11.276 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:11.349 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:11.344+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:11.385 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:11.428 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:10:11.482 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:11.478+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:11.532 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:11.532 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:11.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:11.551 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:11.548+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:11.651 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:11.646+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:11.654 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:11.654 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:11.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:11.752 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:11.748+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:11.809 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:11.810 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:11.818 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:11.853 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:11.846+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:11.915 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:11.915 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:11.924 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:11.954 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:11.950+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:12.024 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.025 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:12.055 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:12.052+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:12.157 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:12.152+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:12.180 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.180 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:12.258 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:12.252+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:12.285 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.285 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:12.359 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:12.356+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:12.405 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.405 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:12.459 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:12.454+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:12.517 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.518 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:12.561 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:12.556+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:12.649 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.649 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:12.662 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:12.656+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:12.762 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:12.758+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:12.768 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.768 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:12.863 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:12.858+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:12.873 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.873 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.882 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:12.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:10:12.964 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:12.960+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:12.980 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:12.980 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:12.989 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:13.039 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:13.059 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:13.065 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:13.058+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:13.079 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:13.101 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:13.125 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:13.130 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:13.130 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:13.130 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:13.130 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:13.139 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:13.147 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:13.165 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:13.158+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:13.172 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:13.172 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:10:13.172 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:13.172 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:13.173 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:13.173 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:13.173 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:13.173 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:13.173 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:13.176 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:10:13.176 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:10:13.241 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:13.241 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:13.242 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:13.242 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:13.242 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:13.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:13.266 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:13.260+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:13.355 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:13.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.52040495, 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:10:13.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:10:13.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.520388611, 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:10:13.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:10:13.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.520363446, 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:10:13.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.51945947900001, 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:13.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:10:13.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:10:13.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.51945014, 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:10:13.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.384 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.51916001799999, 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.385 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:10:13.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:10:13.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:10:13.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:10:13.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:10:13.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.518941324, 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:10:13.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.51254774500001, 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:10:13.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.400 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.508636182, 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:13.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.405 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.508604977, 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:13.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:10:13.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.507785537, 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:10:13.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:10:13.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:10:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.507740728, 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.416 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:13.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:10:13.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:10:13.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.504731474, 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:10:13.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.504714514, 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:10:13.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.504703018, 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:10:13.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.436 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.49995610800001, 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.437 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:13.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:10:13.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:10:13.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.492323477, 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.443 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:10:13.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.448 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.49231127, 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:13.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:10:13.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.453 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.49005635499999, 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:10:13.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:10:13.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:10:13.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:13.459 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 114.489832104, 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:10:13.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:10:13.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.465 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:13.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:10:13.467 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:13.467 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:13.460+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:13.567 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:13.562+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:13.606 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:13.602+0000 7f0ddd714540 -1 osd.0 129 log_to_monitors true 2024-09-16T14:10:13.625 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:13.625 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:13.625 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:13.625 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:13.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:13.668 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:13.662+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:13.732 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:13.732 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:13.732 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:13.732 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:13.732 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:13.741 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:13.787 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:13.784+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:13.845 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:13.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.467484816, 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:10:13.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:13.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.463553398, 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.463529206, 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.855 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:10:13.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.459809935, 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:10:13.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.864 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.45589317, 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:10:13.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:10:13.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:10:13.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.451510891, 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.869 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.45148705, 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:13.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:10:13.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.877 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.45147186200001, 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:10:13.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:10:13.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.451457276, 2024-09-16T14:10:13.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:10:13.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.451353626, 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:10:13.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.891 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.4513259, 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:13.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:10:13.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.44862557, 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.896 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:13.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.448585403, 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.44849568799999, 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.905 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:10:13.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.43988933, 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:10:13.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:10:13.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:13.915 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.43772103800001, 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:10:13.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:10:13.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.288159577000002, 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.921 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:10:13.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [watch reconnect cookie 140383155395456 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:10:13.927 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.158150349000003, 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.013377462, 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:13.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.694814+0000", 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003026674 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.695138+0000", 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032383800000000001 2024-09-16T14:10:13.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697164+0000", 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020260149999999999 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.152287215000001, 2024-09-16T14:10:13.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.007999884, 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:10:13.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696135+0000", 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003762636 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696215+0000", 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0119999999999999e-05 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697528+0000", 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001312887 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697582+0000", 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4409e-05 2024-09-16T14:10:13.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697649+0000", 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7026000000000005e-05 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:201111 3.12 3:4da97782:::rbd_header.11369f5c332a:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e172)", 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.5331123469999994, 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81574065299999998, 2024-09-16T14:10:13.939 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 201111 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308827+0000", 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8559999999999998e-06 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308839+0000", 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2170999999999999e-05 2024-09-16T14:10:13.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308872+0000", 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3034999999999998e-05 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308888+0000", 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5158e-05 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308987+0000", 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8924000000000004e-05 2024-09-16T14:10:13.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.309001+0000", 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4504e-05 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.123915+0000", 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.814914052 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.124090+0000", 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017510599999999999 2024-09-16T14:10:13.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.124565+0000", 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047484699999999998 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:13.944 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:13.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:13.946 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:13.942+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:14.047 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:14.042+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:14.053 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:14.053 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:14.054 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:14.054 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:14.063 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:14.148 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:14.144+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:14.162 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:14.163 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:14.163 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:14.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:14.248 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:14.244+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:14.274 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:10:14.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 172.86010518800001, 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:10:14.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:10:14.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 145.02214286500001, 2024-09-16T14:10:14.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:10:14.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:10:14.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:14.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.494903877, 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:14.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.494880646, 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.49485769899999, 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:10:14.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:10:14.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.49484119, 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:14.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.494823564, 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:10:14.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.48890781999999, 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.310 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:14.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:10:14.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.488890415, 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:10:14.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:10:14.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.488536049, 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:10:14.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.48539219, 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:14.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.48537690400001, 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.328 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:14.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:14.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.48536049499999, 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:14.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:10:14.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.336 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.48528529799999, 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:10:14.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:10:14.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.485187751, 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:14.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.48516250999999, 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:10:14.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:14.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.485133456, 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:10:14.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.481869467, 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:14.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:10:14.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.48185126600001, 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:10:14.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 115.481812088, 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:14.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:14.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:14.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:10:14.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:10:14.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:14.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:14.370 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:14.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:14.372 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:14.368+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:14.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:10:14.453 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:10:14.473 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:14.468+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:14.495 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:14.490+0000 7f0dd1c57700 -1 osd.0 183 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:10:14.504 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:14.504 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:14.504 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:14.504 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:14.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:14.573 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:14.568+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:14.579 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:14.579 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:14.579 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:14.579 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:14.618 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:14.618 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:14.619 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:14.619 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:14.619 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:14.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:14.674 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:14.670+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:14.727 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:14.727 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:14.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:14.727 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:14.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.727 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 107.48828348399999, 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:10:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128121441999994, 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:10:14.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128113091000003, 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.737 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:14.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:10:14.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:10:14.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128080406999999, 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:14.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128071169999998, 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:10:14.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:10:14.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128062638000003, 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.751 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:10:14.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128054061, 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:14.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128044700000004, 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:10:14.760 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:10:14.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128036588000001, 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:10:14.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128024885000002, 2024-09-16T14:10:14.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:10:14.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.773 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128016271999996, 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:10:14.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.128008166000001, 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.778 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:10:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.127999824, 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:10:14.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.127991285999997, 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:14.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.791 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.127982954000004, 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:10:14.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:10:14.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.127974711999997, 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.796 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:10:14.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.800 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.127965149000005, 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:10:14.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.127955768000007, 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:10:14.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:14.809 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.127946127000001, 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:10:14.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:201147 3.1d 3:b85ce93f:::rbd_header.11367eb52311:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.5676438360000002, 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22849376599999999, 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 201147 2024-09-16T14:10:14.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:14.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157189+0000", 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.227e-06 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157199+0000", 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9469999999999998e-06 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157219+0000", 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0035e-05 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157228+0000", 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9369999999999993e-06 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157310+0000", 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1459999999999996e-05 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157322+0000", 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2395000000000001e-05 2024-09-16T14:10:14.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385317+0000", 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.227994851 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385438+0000", 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121371 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:14.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385681+0000", 2024-09-16T14:10:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024254300000000001 2024-09-16T14:10:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:14.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:14.819 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:14.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:10:14.821 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:14.814+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:14.921 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:14.916+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:14.969 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:14.969 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:14.969 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:14.969 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:14.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:10:15.022 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:15.016+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:15.079 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:15.079 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:15.079 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:15.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:10:15.123 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:15.120+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:15.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:10:15.190 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2024-09-16T14:10:15.191 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:15.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:15.246 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:15.242+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:15.305 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:15.322 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:15.325 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:15.320+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:15.341 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:15.346 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:15.346 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:15.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:15.371 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:15.406 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:15.426 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:15.422+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:15.428 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:15.453 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:15.453 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:15.455 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:15.455 INFO:teuthology.orchestra.run.smithi079.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-16T14:10:15.455 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:15.455 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:15.455 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:15.455 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:15.456 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:15.456 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:15.456 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:15.462 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:10:15.463 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:10:15.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:15.527 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:15.524+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:15.601 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:15.601 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:15.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:15.627 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:15.622+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:15.709 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:15.709 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:15.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:15.728 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:15.724+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:15.819 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:15.819 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:15.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:15.830 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:15.822+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:15.930 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:15.924+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:15.930 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:15.930 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:15.939 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:16.031 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:16.030+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:16.047 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.047 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.056 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:16.132 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:16.130+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:16.159 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.159 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:16.232 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:16.226+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:16.267 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.268 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.277 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:16.333 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:16.328+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:16.381 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.381 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.390 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:16.435 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:16.432+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:16.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:10:16.494 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.494 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:16.535 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:16.530+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:16.598 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.598 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:16.635 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:16.632+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:16.711 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.711 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:16.736 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:16.730+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:16.817 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.817 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.826 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:16.871 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:16.868+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:16.922 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:16.923 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:16.932 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:16.938 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:16.932+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:17.027 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:17.027 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:17.036 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:17.039 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:17.036+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:17.135 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:17.135 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:17.139 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:17.136+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:17.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:17.241 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:17.236+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:17.287 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:17.287 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:17.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:17.346 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:17.342+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:17.405 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:17.405 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:17.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:17.442 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:17.436+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:17.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:10:17.476 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:10:17.520 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:17.520 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:17.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:17.574 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:17.570+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:17.598 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:10:17.622 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:17.643 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:17.638+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:17.645 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:17.669 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:17.695 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 110.456740995, 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096578953000005, 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:10:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096570602, 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:10:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:10:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:10:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:10:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096537917999996, 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:10:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096528680999995, 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:10:17.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096520149, 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:10:17.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:10:17.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096511571999997, 2024-09-16T14:10:17.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:10:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.731 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096502211000001, 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.732 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:10:17.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:17.736 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096494098999997, 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:17.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.740 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096482395999999, 2024-09-16T14:10:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:10:17.741 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096473782999993, 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.746 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:10:17.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096465676999998, 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:17.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096457334999997, 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:10:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:10:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:10:17.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096448796999994, 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.761 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:10:17.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:10:17.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096440465000001, 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:10:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:10:17.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:17.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096432222999994, 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:10:17.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096422660000002, 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:10:17.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:10:17.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096413279000004, 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.780 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:10:17.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.784 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 87.096403637999998, 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:17.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:10:17.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:201147 3.1d 3:b85ce93f:::rbd_header.11367eb52311:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 12.536101347000001, 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22849376599999999, 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 201147 2024-09-16T14:10:17.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157189+0000", 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.227e-06 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157199+0000", 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9469999999999998e-06 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157219+0000", 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0035e-05 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157228+0000", 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9369999999999993e-06 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157310+0000", 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1459999999999996e-05 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157322+0000", 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2395000000000001e-05 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385317+0000", 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.227994851 2024-09-16T14:10:17.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385438+0000", 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121371 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385681+0000", 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024254300000000001 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:17.796 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:17.796 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:17.797 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:17.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:10:17.798 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:17.798 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:17.799 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:17.799 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-16T14:10:17.799 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:17.799 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:17.800 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:17.800 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:17.800 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:17.800 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:17.800 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:17.800 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:17.790+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:17.898 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:17.894+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:17.956 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:17.956 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:17.966 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:10:18.010 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:18.006+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:18.068 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:18.069 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:18.069 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:18.069 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:18.069 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:18.078 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:10:18.099 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:18.094+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:18.179 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:3369 3.e e185/184 3:75d984b1:::rbd_data.1136f7114f14.0000000000000111:18 v 185'454, mlcod=185'450)", 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:16.624798+0000", 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.551398238, 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057993660000000002, 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624798+0000", 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624798+0000", 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624799+0000", 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5199999999999996e-07 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624806+0000", 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0339999999999999e-06 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624822+0000", 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6892e-05 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624829+0000", 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1929999999999998e-06 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624879+0000", 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0185999999999997e-05 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624904+0000", 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5134999999999999e-05 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.630549+0000", 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056451519999999996 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.630597+0000", 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8022000000000002e-05 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:3370 3.e e185/184 3:72a14c44:::rbd_data.113644f64486.0000000000000002:23 v 185'456, mlcod=185'452)", 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:16.638190+0000", 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.5380062720000001, 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037652039999999999, 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.638190+0000", 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.638190+0000", 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.638191+0000", 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2200000000000003e-07 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.638201+0000", 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0581000000000001e-05 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.638216+0000", 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4412e-05 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.638225+0000", 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7200000000000001e-06 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.638292+0000", 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6940999999999995e-05 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.638324+0000", 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2129999999999999e-05 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.641898+0000", 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035731729999999998 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.641955+0000", 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7425e-05 2024-09-16T14:10:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:3375 3.e e185/184 3:706349b2:::rbd_data.1136bef3a187.0000000000000001:2a v 185'466, mlcod=185'462)", 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:16.655444+0000", 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.5207516029999999, 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035556070000000001, 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.655444+0000", 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.655444+0000", 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.655445+0000", 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1999999999999998e-07 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.655454+0000", 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.585e-06 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.655467+0000", 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2656999999999999e-05 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.655473+0000", 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.263e-06 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.655528+0000", 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4765999999999997e-05 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.655554+0000", 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6775999999999999e-05 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.658947+0000", 2024-09-16T14:10:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033926939999999999 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.659000+0000", 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3046e-05 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:3376 3.e e185/184 3:736a348d:::rbd_data.11366c63f9a8.0000000000000001:2e v 185'468, mlcod=185'464)", 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:16.662583+0000", 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.513612615, 2024-09-16T14:10:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039302540000000002, 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.662583+0000", 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.662583+0000", 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.662584+0000", 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.032e-06 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.662594+0000", 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5429999999999999e-06 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.662608+0000", 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3927999999999999e-05 2024-09-16T14:10:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.662615+0000", 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2640000000000002e-06 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.662924+0000", 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00030917999999999999 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.662955+0000", 2024-09-16T14:10:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0843999999999998e-05 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.666404+0000", 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034484060000000002 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.666514+0000", 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011005699999999999 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203339 3.1c e186/185 3:3836cdaa:::rbd_data.11362e0d5ebf.000000000000000d:head v 186'976, mlcod=132'758)", 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.671281+0000", 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.50491515200000003, 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047611839999999999, 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.671281+0000", 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.671281+0000", 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.671282+0000", 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0999999999999997e-07 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.671290+0000", 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8339999999999999e-06 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.671307+0000", 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7405999999999999e-05 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.671315+0000", 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4290000000000001e-06 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.671384+0000", 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.8725000000000004e-05 2024-09-16T14:10:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.671407+0000", 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2317999999999999e-05 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.675968+0000", 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045613420000000003 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676042+0000", 2024-09-16T14:10:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4220000000000004e-05 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203347 3.e e186/184 3:735e1e72:::rbd_data.11362e0d5ebf.0000000000000015:head v 186'471, mlcod=185'468)", 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.674776+0000", 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.501420541, 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0047364379999999999, 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.674776+0000", 2024-09-16T14:10:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.674776+0000", 2024-09-16T14:10:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.674776+0000", 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9299999999999997e-07 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.674784+0000", 2024-09-16T14:10:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2540000000000003e-06 2024-09-16T14:10:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.674799+0000", 2024-09-16T14:10:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5798999999999999e-05 2024-09-16T14:10:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.674805+0000", 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7579999999999998e-06 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.674887+0000", 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2367999999999995e-05 2024-09-16T14:10:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.674906+0000", 2024-09-16T14:10:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8666000000000001e-05 2024-09-16T14:10:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.679470+0000", 2024-09-16T14:10:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045640999999999998 2024-09-16T14:10:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.679512+0000", 2024-09-16T14:10:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1699999999999997e-05 2024-09-16T14:10:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203348 3.3 e186/185 3:c6d65799:::rbd_data.11362e0d5ebf.0000000000000016:head v 186'1312, mlcod=134'982)", 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.676222+0000", 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.49997386500000002, 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059874780000000001, 2024-09-16T14:10:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676222+0000", 2024-09-16T14:10:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676222+0000", 2024-09-16T14:10:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676223+0000", 2024-09-16T14:10:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.61e-07 2024-09-16T14:10:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676229+0000", 2024-09-16T14:10:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3709999999999996e-06 2024-09-16T14:10:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676243+0000", 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3326000000000001e-05 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676248+0000", 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6670000000000001e-06 2024-09-16T14:10:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676287+0000", 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8792999999999999e-05 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676312+0000", 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5111999999999999e-05 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.682174+0000", 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058622199999999996 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.682210+0000", 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5327999999999998e-05 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203352 3.e e186/184 3:75cd9285:::rbd_data.11362e0d5ebf.000000000000001a:head v 186'472, mlcod=185'468)", 2024-09-16T14:10:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.678182+0000", 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.49801366699999999, 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040229230000000003, 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.678182+0000", 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.678182+0000", 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.678183+0000", 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.8599999999999997e-07 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.678190+0000", 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1420000000000004e-06 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.678201+0000", 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1187e-05 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.678209+0000", 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9619999999999993e-06 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.678254+0000", 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4727000000000001e-05 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.678273+0000", 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8587e-05 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.682162+0000", 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038888870000000002 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.682205+0000", 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3645e-05 2024-09-16T14:10:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203370 3.1b e186/185 3:d8ad2d45:::rbd_data.11362e0d5ebf.000000000000002c:head v 186'687, mlcod=0'0)", 2024-09-16T14:10:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.685914+0000", 2024-09-16T14:10:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.49028234599999998, 2024-09-16T14:10:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037548450000000001, 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685914+0000", 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685914+0000", 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685914+0000", 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2500000000000001e-07 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685919+0000", 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0239999999999997e-06 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685931+0000", 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2207e-05 2024-09-16T14:10:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685938+0000", 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0430000000000002e-06 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.686000+0000", 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1803000000000004e-05 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.686021+0000", 2024-09-16T14:10:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0664000000000001e-05 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.689628+0000", 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003607014 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.689669+0000", 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0765e-05 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203369 3.c e186/185 3:332530ad:::rbd_data.11362e0d5ebf.000000000000002b:head v 186'604, mlcod=0'0)", 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.685914+0000", 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.49028204600000003, 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038433999999999999, 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685914+0000", 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685914+0000", 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685914+0000", 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685919+0000", 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0549999999999999e-06 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685931+0000", 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1174e-05 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685939+0000", 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2849999999999995e-06 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.685999+0000", 2024-09-16T14:10:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0126e-05 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.686021+0000", 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1583999999999999e-05 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.689682+0000", 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003661063 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.689757+0000", 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5747000000000005e-05 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203377 3.f e186/185 3:f4b331dc:::rbd_data.11362e0d5ebf.0000000000000033:head v 186'943, mlcod=131'775)", 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.688761+0000", 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.48743493599999999, 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003867901, 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.688761+0000", 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.688761+0000", 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.688762+0000", 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2399999999999996e-07 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.688769+0000", 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4109999999999999e-06 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.688784+0000", 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4341e-05 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.688790+0000", 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5479999999999998e-06 2024-09-16T14:10:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.688836+0000", 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5670999999999997e-05 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.688861+0000", 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5143999999999998e-05 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.692581+0000", 2024-09-16T14:10:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0037199590000000001 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.692629+0000", 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7902999999999999e-05 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203381 3.1b e186/185 3:db0f5094:::rbd_data.11362e0d5ebf.0000000000000037:head v 186'688, mlcod=134'457)", 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.690689+0000", 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.48550722299999999, 2024-09-16T14:10:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070459449999999996, 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690689+0000", 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690689+0000", 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690690+0000", 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9100000000000003e-07 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690696+0000", 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6560000000000003e-06 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690713+0000", 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6725999999999999e-05 2024-09-16T14:10:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690720+0000", 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7510000000000002e-06 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690993+0000", 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027328299999999998 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691020+0000", 2024-09-16T14:10:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7299000000000001e-05 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.697683+0000", 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066627839999999997 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.697735+0000", 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1654999999999999e-05 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203383 3.1b e186/185 3:df8db72c:::rbd_data.11362e0d5ebf.0000000000000039:head v 186'689, mlcod=134'457)", 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.691677+0000", 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.48451940599999999, 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060938829999999996, 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691677+0000", 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691677+0000", 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691677+0000", 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3799999999999997e-07 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691682+0000", 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0669999999999999e-06 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691693+0000", 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0710000000000001e-05 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691697+0000", 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2300000000000002e-06 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691730+0000", 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2823000000000001e-05 2024-09-16T14:10:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691746+0000", 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5982999999999999e-05 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.697739+0000", 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059933950000000003 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.697771+0000", 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1037000000000001e-05 2024-09-16T14:10:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203390 3.f e186/185 3:f202c33e:::rbd_data.11362e0d5ebf.0000000000000040:head v 186'944, mlcod=131'775)", 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.698181+0000", 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.478015305, 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050231950000000003, 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.698181+0000", 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.698181+0000", 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.698182+0000", 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6099999999999999e-07 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.698189+0000", 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9269999999999999e-06 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.698201+0000", 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2319e-05 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.698206+0000", 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0180000000000001e-06 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.698247+0000", 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0706e-05 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.698280+0000", 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3643999999999998e-05 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.703157+0000", 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004876432 2024-09-16T14:10:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.703204+0000", 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7188000000000003e-05 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203393 3.1d e186/185 3:bc6bf59c:::rbd_data.11362e0d5ebf.0000000000000043:head v 186'640, mlcod=0'0)", 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.699744+0000", 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.47645244799999997, 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063103760000000004, 2024-09-16T14:10:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699744+0000", 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699744+0000", 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699744+0000", 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7300000000000005e-07 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699752+0000", 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5449999999999998e-06 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699763+0000", 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226999999999999e-05 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699768+0000", 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0170000000000004e-06 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699818+0000", 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9452e-05 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699838+0000", 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0863999999999999e-05 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.705984+0000", 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061452750000000004 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.706054+0000", 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0222999999999995e-05 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203395 3.c e186/185 3:35b3bf1b:::rbd_data.11362e0d5ebf.0000000000000045:head v 186'605, mlcod=171'548)", 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.701609+0000", 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.47458749099999997, 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0049061570000000004, 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.701609+0000", 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.701609+0000", 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.701610+0000", 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.701617+0000", 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3189999999999997e-06 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.701635+0000", 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8598e-05 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.701643+0000", 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3050000000000004e-06 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.701750+0000", 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010688600000000001 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.701778+0000", 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8843000000000002e-05 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.706262+0000", 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0044830699999999996 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.706515+0000", 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00025317800000000001 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203399 3.1b e186/185 3:dc631e35:::rbd_data.11362e0d5ebf.0000000000000049:head v 186'690, mlcod=134'457)", 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.702185+0000", 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.47401090200000001, 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039806759999999998, 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.702185+0000", 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.702185+0000", 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.702186+0000", 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4200000000000005e-07 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.702195+0000", 2024-09-16T14:10:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7110000000000001e-06 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.702207+0000", 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2781e-05 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.702215+0000", 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1829999999999997e-06 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.702255+0000", 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0219999999999998e-05 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.702288+0000", 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2917999999999999e-05 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.706135+0000", 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0038472379999999998 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.706166+0000", 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0783e-05 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203424 3.13 e186/185 3:c92aa426:::rbd_data.11362e0d5ebf.0000000000000062:head v 186'736, mlcod=0'0)", 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.714254+0000", 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.46194251200000003, 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0053503700000000001, 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.714254+0000", 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.714254+0000", 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.714254+0000", 2024-09-16T14:10:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3499999999999995e-07 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.714263+0000", 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6549999999999993e-06 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.714275+0000", 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.202e-05 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.714281+0000", 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9769999999999997e-06 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.714376+0000", 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5168999999999996e-05 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.714401+0000", 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5335000000000001e-05 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.719562+0000", 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0051604470000000003 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.719604+0000", 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2032e-05 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203425 3.f e186/185 3:f1ba6ca4:::rbd_data.11362e0d5ebf.0000000000000063:head v 186'945, mlcod=131'775)", 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.715267+0000", 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.46092881299999999, 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.004553161, 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.715267+0000", 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.715267+0000", 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.715268+0000", 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3499999999999995e-07 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.715275+0000", 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.2609999999999996e-06 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.715292+0000", 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6453e-05 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.715299+0000", 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3599999999999998e-06 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.715388+0000", 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9169e-05 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.715410+0000", 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1979000000000001e-05 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.719776+0000", 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0043656670000000002 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.719820+0000", 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4536999999999997e-05 2024-09-16T14:10:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203428 3.1c e186/185 3:3b8a6629:::rbd_data.11362e0d5ebf.0000000000000066:head v 186'978, mlcod=132'758)", 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.716173+0000", 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 0.46002281299999997, 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034801810000000002, 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.716173+0000", 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.716173+0000", 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.716174+0000", 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1699999999999997e-07 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.716181+0000", 2024-09-16T14:10:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4370000000000001e-06 2024-09-16T14:10:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.716191+0000", 2024-09-16T14:10:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8719999999999997e-06 2024-09-16T14:10:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.716196+0000", 2024-09-16T14:10:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1459999999999998e-06 2024-09-16T14:10:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.716239+0000", 2024-09-16T14:10:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3631999999999998e-05 2024-09-16T14:10:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.716260+0000", 2024-09-16T14:10:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1231000000000001e-05 2024-09-16T14:10:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.719615+0000", 2024-09-16T14:10:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0033543499999999999 2024-09-16T14:10:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.719653+0000", 2024-09-16T14:10:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8695999999999997e-05 2024-09-16T14:10:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.276 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:18.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:18.278 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:18.272+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:18.379 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:18.376+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:18.394 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:18.395 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:18.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:18.479 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:18.474+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:18.513 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:18.514 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:18.523 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:18.581 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:18.576+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:18.622 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71532560700001, 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:10:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:10:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:10:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:10:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:10:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71529266600001, 2024-09-16T14:10:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:10:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:10:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71526798299999, 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:10:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.715244754, 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:10:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:10:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:10:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71335592599999, 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:10:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:10:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:10:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71189630800001, 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:10:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71188053899999, 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:10:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:10:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:10:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:10:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.711864787, 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:10:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71184935599999, 2024-09-16T14:10:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:10:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:10:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:10:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71183371800001, 2024-09-16T14:10:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:10:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:10:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:10:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71181857900001, 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:10:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.711803742, 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:10:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:10:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71178073799999, 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:10:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:10:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71176551299999, 2024-09-16T14:10:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.71175032900001, 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:10:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:10:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:10:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:10:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:10:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:10:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:10:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:10:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:10:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:10:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:10:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:10:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:10:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:10:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:10:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:10:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:10:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:10:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.711735084, 2024-09-16T14:10:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:10:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:10:18.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:18.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:18.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:10:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:10:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:10:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.711719926, 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:10:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:10:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:10:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:10:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:10:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:10:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200434 3.11 3:8f4127d4:::rbd_data.113619b3ee73.0000000000000003:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.520663228, 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0847031039999999, 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200434 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098242+0000", 2024-09-16T14:10:18.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.39e-06 2024-09-16T14:10:18.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098250+0000", 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0870000000000007e-06 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098270+0000", 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9760999999999999e-05 2024-09-16T14:10:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098281+0000", 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1258e-05 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.099140+0000", 2024-09-16T14:10:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085896699999999998 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.099150+0000", 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0516999999999999e-05 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180747+0000", 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0815962020000001 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180857+0000", 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011097199999999999 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,3", 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180980+0000", 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012213400000000001 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.181989+0000", 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001009338 2024-09-16T14:10:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182283+0000", 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029419 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182697+0000", 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041376599999999998 2024-09-16T14:10:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182774+0000", 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6965000000000006e-05 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182942+0000", 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016855700000000001 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200437 3.a 3:5303f819:::rbd_data.113619b3ee73.0000000000000006:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.520446892999999, 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0809326290000001, 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200437 2024-09-16T14:10:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098457+0000", 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9600000000000008e-07 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098462+0000", 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4070000000000001e-06 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098476+0000", 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4243999999999999e-05 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098486+0000", 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3950000000000007e-06 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098546+0000", 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0717999999999997e-05 2024-09-16T14:10:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098558+0000", 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1089e-05 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173098+0000", 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0745404629999999 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173311+0000", 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021308399999999999 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173553+0000", 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000241964 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.177396+0000", 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00384337 2024-09-16T14:10:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179118+0000", 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017211419999999999 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179185+0000", 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7290000000000001e-05 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179388+0000", 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020346700000000001 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200439 3.a 3:53cece2e:::rbd_data.113619b3ee73.0000000000000008:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 18.520407424999998, 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0811117130000001, 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200439 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098496+0000", 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7000000000000004e-07 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098499+0000", 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2710000000000001e-06 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098509+0000", 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0066e-05 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098515+0000", 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098588+0000", 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3070000000000003e-05 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098596+0000", 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9880000000000004e-06 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174400+0000", 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0758035370000001 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174548+0000", 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014881900000000001 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174732+0000", 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018332799999999999 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.178747+0000", 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040152809999999999 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179446+0000", 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00069927300000000004 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179505+0000", 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8573999999999999e-05 2024-09-16T14:10:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179607+0000", 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101951 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:18.757 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:18.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:18.759 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:18.754+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:18.860 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:18.854+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:18.873 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:18.873 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:18.873 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:18.874 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:18.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:18.961 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:18.954+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:19.043 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:19.043 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:19.043 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:19.043 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:19.043 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:19.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:19.098 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:19.094+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:19.157 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:19.157 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:19.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:19.157 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 107.635174851, 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:10:19.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:10:19.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:10:19.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:10:19.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:10:19.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:10:19.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 97.629903511999998, 2024-09-16T14:10:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:19.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:10:19.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:10:19.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:10:19.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:10:19.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:10:19.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:10:19.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:19.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:10:19.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:10:19.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:19.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:10:19.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:10:19.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:10:19.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:10:19.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.447394997999993, 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:10:19.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.443994466999996, 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.179 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:19.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:10:19.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:10:19.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.439943188000001, 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:10:19.184 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:19.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:10:19.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.436190272000005, 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:10:19.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.193 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.432110743999999, 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:19.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.197 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.432094141999997, 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.198 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:10:19.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.199 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:19.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:10:19.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:10:19.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:10:19.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.432078165999997, 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.204 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:10:19.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:10:19.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:10:19.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.432060214000003, 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:10:19.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.427865083, 2024-09-16T14:10:19.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:10:19.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:10:19.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.427768104999998, 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.218 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.222 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.427743563999996, 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:10:19.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:10:19.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:10:19.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:10:19.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.424186801999994, 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.228 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.424169043999996, 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:10:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:10:19.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.424151381000001, 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:19.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:10:19.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.424134187000007, 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.420437769000003, 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.247 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:10:19.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 88.408107817000001, 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:10:19.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:10:19.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.734595644999999, 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:10:19.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.258 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:19.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:19.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:19.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:19.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:19.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:19.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:19.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:10:19.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:10:19.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:10:19.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:10:19.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:10:19.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:10:19.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:19.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:10:19.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:10:19.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.266 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.266 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:19.266 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:19.266 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:19.267 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:19.268 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:19.264+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:19.368 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:19.362+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:19.372 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:19.372 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:19.372 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:19.381 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:19.469 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:19.464+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:19.483 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:19.483 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:19.483 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:19.483 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:19.484 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:19.493 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:19.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:10:19.570 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:19.564+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:19.589 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 1, 3] out_osds: [7] dead_osds: [] live_osds: [1, 2, 4, 5, 6, 7, 3, 0] 2024-09-16T14:10:19.589 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:10:19.589 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-16T14:10:19.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-16T14:10:19.593 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:19.593 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:19.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:19.593 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:19.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.593 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:19.593 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:19.593 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 182.251230881, 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:10:19.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 182.25120848399999, 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:10:19.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:10:19.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 182.250601773, 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:10:19.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:10:19.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:10:19.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:10:19.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:10:19.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:10:19.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.612 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:19.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:19.613 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 182.25052744600001, 2024-09-16T14:10:19.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:10:19.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:10:19.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:10:19.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:10:19.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:10:19.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:10:19.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:10:19.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 172.279602654, 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:10:19.622 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:10:19.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 168.651644073, 2024-09-16T14:10:19.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:19.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:10:19.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:10:19.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:10:19.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:10:19.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:10:19.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:10:19.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 168.12148360699999, 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.634 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:10:19.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:10:19.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:10:19.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 140.473935076, 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.641 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:19.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.92752777199999, 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:10:19.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.92749354700001, 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.650 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:10:19.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.92746383100001, 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:19.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:10:19.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.92744416900001, 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:19.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:10:19.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.918684044, 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:10:19.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:10:19.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.833957502, 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:10:19.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:10:19.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:10:19.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.188953964999996, 2024-09-16T14:10:19.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:10:19.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:10:19.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:10:19.680 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 99.188867103999996, 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:10:19.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:19.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:10:19.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:10:19.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:10:19.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:10:19.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:10:19.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:10:19.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:10:19.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:10:19.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:10:19.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:10:19.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.689 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.690 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:10:19.690 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:19.690 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.999560485000003, 2024-09-16T14:10:19.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:10:19.690 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.690 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.691 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:10:19.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.691 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:19.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:19.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:10:19.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:10:19.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:10:19.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:10:19.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.114336262999998, 2024-09-16T14:10:19.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:10:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.699 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:19.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:10:19.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:10:19.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:10:19.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:10:19.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:10:19.705 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.706 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.706 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:10:19.706 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:19.706 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.097273080999997, 2024-09-16T14:10:19.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:10:19.707 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:19.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.707 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.708 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:10:19.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.708 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:19.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:19.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:10:19.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:10:19.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:10:19.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:10:19.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:10:19.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:194522 3.6 3:67cae2ed:::rbd_mirroring:head [call rbd.mirror_image_get in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.700836793999997, 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02278196, 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 194522 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887296+0000", 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887310+0000", 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3406e-05 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909564+0000", 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022254260000000001 2024-09-16T14:10:19.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909597+0000", 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2869e-05 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909755+0000", 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158426 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.910075+0000", 2024-09-16T14:10:19.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031989900000000001 2024-09-16T14:10:19.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:19.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:19.720 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:19.721 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:19.722 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:19.716+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:19.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:10:19.822 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:19.816+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:19.829 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:19.829 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:19.830 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:19.830 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:19.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-09-16T14:10:19.826+0000 7fcedf7fe700 0 --2- 172.21.15.79:0/757907383 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fcee013dba0 0x7fcee014bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T14:10:19.839 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:19.891 INFO:tasks.ceph.mon.a.smithi079.stderr:2024-09-16T14:10:19.886+0000 7f5ada529700 -1 mon.a@0(leader).osd e188 definitely_dead 0 2024-09-16T14:10:19.923 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:19.920+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:19.944 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:19.944 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:19.949 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:10:19.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:19.969 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:19.987 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:20.005 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:20.024 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:20.018+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:20.026 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:20.044 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:20.051 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.051 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:20.065 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:20.084 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:10:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:20.125 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:20.120+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:20.163 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.164 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:20.226 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:20.220+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:20.275 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.275 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.284 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:20.326 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:20.324+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:10:20.389 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.390 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:20.403 INFO:teuthology.orchestra.run.smithi079.stderr:marked in osd.7. 2024-09-16T14:10:20.414 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-16T14:10:20.427 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:20.422+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:20.529 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:20.522+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:20.539 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.540 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.548 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:20.629 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:20.622+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:20.643 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.643 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:20.730 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:20.726+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:20.831 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:20.826+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:20.864 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.864 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:20.933 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:20.926+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:20.971 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:20.971 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:20.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:21.033 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:21.028+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:21.128 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:21.128 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:21.134 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:21.130+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:21.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:10:21.232 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:21.232 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:21.235 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:21.232+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:21.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:10:21.335 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:21.332+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:21.345 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:21.345 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:21.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:10:21.436 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:21.430+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:21.458 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:21.458 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:21.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:21.538 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:21.532+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:21.562 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:21.562 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:21.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:21.639 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:21.634+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:21.666 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:21.667 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:21.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:21.740 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:21.734+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:21.822 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:21.822 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:21.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:21.841 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:21.836+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:21.930 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:21.930 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:21.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:21.942 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:21.934+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:22.039 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:22.039 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:22.042 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:22.036+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:22.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:22.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:10:22.143 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:22.138+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:22.145 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:22.145 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:22.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:22.228 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:10:22.243 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:22.238+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:22.250 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:22.261 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:22.262 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:22.271 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:22.272 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:22.290 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:22.314 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:22.335 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:22.344 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:22.340+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:22.359 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:22.373 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:22.373 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:22.373 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:22.373 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:22.373 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:22.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:22.385 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:22.388 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:10:22.388 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:22.388 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:22.388 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:22.388 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:22.388 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:22.388 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:22.388 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:22.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:10:22.450 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:22.446+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1144 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 185.15514680999999, 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021585191, 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1144 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336899+0000", 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-07 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336903+0000", 2024-09-16T14:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336909+0000", 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8279999999999997e-06 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336914+0000", 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.350946+0000", 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014032371 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351018+0000", 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2201999999999998e-05 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351407+0000", 2024-09-16T14:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038867799999999999 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355675+0000", 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042678539999999997 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358390+0000", 2024-09-16T14:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00271521 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358428+0000", 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8047999999999999e-05 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358484+0000", 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6218999999999998e-05 2024-09-16T14:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1145 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 185.15512441300001, 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021668329, 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1145 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336921+0000", 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336922+0000", 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7700000000000004e-07 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336925+0000", 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9720000000000001e-06 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336933+0000", 2024-09-16T14:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5180000000000004e-06 2024-09-16T14:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.336938+0000", 2024-09-16T14:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2270000000000004e-06 2024-09-16T14:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351798+0000", 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014859931 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.351897+0000", 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9244000000000001e-05 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.352328+0000", 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043156299999999999 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.355713+0000", 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003384162 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358497+0000", 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027845449999999998 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358540+0000", 2024-09-16T14:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2701000000000003e-05 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358590+0000", 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889000000000003e-05 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1146 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 185.15451770199999, 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021154976999999998, 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1146 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337528+0000", 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337530+0000", 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5319999999999999e-06 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337540+0000", 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0173999999999999e-05 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337555+0000", 2024-09-16T14:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5614e-05 2024-09-16T14:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337564+0000", 2024-09-16T14:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5830000000000007e-06 2024-09-16T14:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353033+0000", 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015468922 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353108+0000", 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4696000000000004e-05 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353417+0000", 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030942200000000001 2024-09-16T14:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357526+0000", 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041085549999999998 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358601+0000", 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010755179999999999 2024-09-16T14:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358634+0000", 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2923999999999999e-05 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.358683+0000", 2024-09-16T14:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9036999999999998e-05 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:1147 3.1b 3:de203a46:::rbd_object_map.11365ca8c0d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 185.154443375, 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024656404, 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1147 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337602+0000", 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337603+0000", 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337610+0000", 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337624+0000", 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3558e-05 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.337631+0000", 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4739999999999999e-06 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353818+0000", 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016187041999999999 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.353916+0000", 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7575e-05 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.354426+0000", 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00051033100000000004 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.357552+0000", 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031264280000000001 2024-09-16T14:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362053+0000", 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045007119999999996 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362140+0000", 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6577999999999997e-05 2024-09-16T14:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:17.362259+0000", 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118997 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4406.0:9779 3.7 e39/34 3:e524d97b:::rbd_data.1136a719415e.0000000000000001:head v 39'46, mlcod=39'44)", 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 175.18351858299999, 2024-09-16T14:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.027526115, 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308527+0000", 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.308530+0000", 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2310000000000001e-06 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319025+0000", 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010494408 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319065+0000", 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0259000000000002e-05 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319083+0000", 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8184000000000001e-05 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319214+0000", 2024-09-16T14:10:22.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013061299999999999 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.319281+0000", 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7321000000000001e-05 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336004+0000", 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016722504999999999 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:27.336053+0000", 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9594e-05 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12321 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:22.528 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 171.55556000199999, 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021502953000000002, 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12321 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:22.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936486+0000", 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936489+0000", 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2200000000000001e-06 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936508+0000", 2024-09-16T14:10:22.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9110000000000002e-05 2024-09-16T14:10:22.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936548+0000", 2024-09-16T14:10:22.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9544e-05 2024-09-16T14:10:22.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936563+0000", 2024-09-16T14:10:22.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5509999999999999e-05 2024-09-16T14:10:22.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936693+0000", 2024-09-16T14:10:22.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000130316 2024-09-16T14:10:22.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.936938+0000", 2024-09-16T14:10:22.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024443700000000002 2024-09-16T14:10:22.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:10:22.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.944431+0000", 2024-09-16T14:10:22.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074931750000000004 2024-09-16T14:10:22.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957728+0000", 2024-09-16T14:10:22.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013296944 2024-09-16T14:10:22.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957836+0000", 2024-09-16T14:10:22.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107711 2024-09-16T14:10:22.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957926+0000", 2024-09-16T14:10:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0420999999999994e-05 2024-09-16T14:10:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:30.957989+0000", 2024-09-16T14:10:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2564999999999995e-05 2024-09-16T14:10:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:12568 3.6 3:669f02f6:::rbd_data.11364aaed44f.0000000000000006:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 171.02539953600001, 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.043133797000000001, 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12568 2024-09-16T14:10:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466646+0000", 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466648+0000", 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466660+0000", 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1907e-05 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466697+0000", 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6609999999999997e-05 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466710+0000", 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2614e-05 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.466857+0000", 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014777200000000001 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.467004+0000", 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146272 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.478597+0000", 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011592899 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509591+0000", 2024-09-16T14:10:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.030994609999999999 2024-09-16T14:10:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509615+0000", 2024-09-16T14:10:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4337000000000001e-05 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509712+0000", 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6317999999999995e-05 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:31.509780+0000", 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8204e-05 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:83994 3.9 3:922d6441:::rbd_data.113618577e70.0000000000000033:17 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 143.377851005, 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023631915, 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:22.547 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 83994 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114195+0000", 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3200000000000005e-07 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114198+0000", 2024-09-16T14:10:22.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.157e-06 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114213+0000", 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4758e-05 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.114219+0000", 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5459999999999999e-06 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129082+0000", 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014863229 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.129164+0000", 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1848000000000006e-05 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:59.137827+0000", 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086628450000000006 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101796 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:22.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 138.83144370100001, 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020215466000000001, 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101796 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660602+0000", 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660603+0000", 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1099999999999995e-07 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660610+0000", 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660626+0000", 2024-09-16T14:10:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5633000000000001e-05 2024-09-16T14:10:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660631+0000", 2024-09-16T14:10:22.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2299999999999999e-06 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680628+0000", 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019997187 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680716+0000", 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7362999999999997e-05 2024-09-16T14:10:22.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680818+0000", 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101885 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101797 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 198656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 138.831409476, 2024-09-16T14:10:22.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020400880999999999, 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101797 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660636+0000", 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660637+0000", 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2200000000000004e-07 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660639+0000", 2024-09-16T14:10:22.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5150000000000001e-06 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660656+0000", 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6653000000000001e-05 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660661+0000", 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1660000000000002e-06 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680845+0000", 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020183856999999999 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.680929+0000", 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3967000000000004e-05 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681037+0000", 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108101 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.562 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101798 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 138.83137976, 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020591387999999999, 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101798 2024-09-16T14:10:22.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660666+0000", 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1399999999999997e-07 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660669+0000", 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.351e-06 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660677+0000", 2024-09-16T14:10:22.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7170000000000008e-06 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660681+0000", 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2660000000000003e-06 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681067+0000", 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020385698000000001 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681160+0000", 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3843000000000003e-05 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681257+0000", 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6998999999999997e-05 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:101799 3.6 3:61c495da:::rbd_data.113618577e70.000000000000001a:17 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:22.567 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 138.831360098, 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020795574000000001, 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 101799 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660686+0000", 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4900000000000001e-07 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660688+0000", 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.063e-06 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660695+0000", 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919999999999998e-06 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.660699+0000", 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1899999999999997e-06 2024-09-16T14:10:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681292+0000", 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02059267 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681376+0000", 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4104000000000007e-05 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.681481+0000", 2024-09-16T14:10:22.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105606 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:102335 3.6 3:62860d3d:::rbd_data.113619b3ee73.0000000000000001:16 [read 94208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 138.822599973, 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021330890000000002, 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 102335 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669446+0000", 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669448+0000", 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1840000000000002e-06 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669455+0000", 2024-09-16T14:10:22.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7909999999999999e-06 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.669459+0000", 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1930000000000003e-06 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686444+0000", 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016984546999999999 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.686543+0000", 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8647000000000006e-05 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:03.690777+0000", 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042341360000000003 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138236 3.6 3:63f5e823:::rbd_data.113662e21c28.0000000000000009:1e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 123.737873431, 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020081209999999999, 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138236 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754172+0000", 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754173+0000", 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754175+0000", 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9970000000000001e-06 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754181+0000", 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.015e-06 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.754184+0000", 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6550000000000002e-06 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769067+0000", 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014882111999999999 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.769150+0000", 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3455000000000006e-05 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.774254+0000", 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051035220000000001 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.580 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153524 3.6 3:61ae2d85:::rbd_data.1136a570af09.0000000000000000:head [truncate 327680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.092869894, 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.967773192, 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.581 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153524 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399176+0000", 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399179+0000", 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7360000000000001e-06 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399190+0000", 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1501000000000001e-05 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399223+0000", 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2994999999999998e-05 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399234+0000", 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0542e-05 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399344+0000", 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010990899999999999 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399368+0000", 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4362000000000002e-05 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.363953+0000", 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96458496400000004 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364337+0000", 2024-09-16T14:10:22.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038433100000000002 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.364533+0000", 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019566900000000001 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366791+0000", 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022579649999999998 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366862+0000", 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0944000000000004e-05 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366913+0000", 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0933999999999997e-05 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366949+0000", 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6340000000000001e-05 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:153528 3.6 3:631dfd27:::rbd_data.1136a570af09.0000000000000004:head [truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.092783033, 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96879234599999997, 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 153528 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399263+0000", 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399264+0000", 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0300000000000001e-06 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399269+0000", 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2979999999999999e-06 2024-09-16T14:10:22.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399284+0000", 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5392000000000001e-05 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399294+0000", 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399419+0000", 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491600000000001 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:40.399427+0000", 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8709999999999997e-06 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365487+0000", 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.96606006799999999 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365626+0000", 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000139529 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.365769+0000", 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143041 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.366798+0000", 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010284840000000001 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.367971+0000", 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001173374 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368019+0000", 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8105000000000001e-05 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:41.368055+0000", 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5848e-05 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182668 3.6 3:65c9d696:::rbd_data.11366c63f9a8.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 81.903476413999996, 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028516531000000001, 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.595 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182668 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588569+0000", 2024-09-16T14:10:22.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588572+0000", 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588583+0000", 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1001e-05 2024-09-16T14:10:22.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588613+0000", 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9882999999999998e-05 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588625+0000", 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2172e-05 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588728+0000", 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010327600000000001 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.588845+0000", 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116859 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.595745+0000", 2024-09-16T14:10:22.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068995949999999997 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616984+0000", 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021238867000000002 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.616994+0000", 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0607e-05 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617036+0000", 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1826000000000002e-05 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.617086+0000", 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9913999999999997e-05 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:184239 3.6 3:64d8e54f:::rbd_data.1136b1466cf.000000000000002a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 77.018252192000006, 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023635985000000002, 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 184239 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473794+0000", 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473796+0000", 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4830000000000002e-06 2024-09-16T14:10:22.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473819+0000", 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3195999999999999e-05 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473856+0000", 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6464000000000002e-05 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.473874+0000", 2024-09-16T14:10:22.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8099000000000001e-05 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474006+0000", 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000132602 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.474152+0000", 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014547799999999999 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.482389+0000", 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082372509999999993 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497260+0000", 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014871128000000001 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497283+0000", 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2612e-05 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497370+0000", 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7570000000000004e-05 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:05.497430+0000", 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9101999999999998e-05 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185252 3.2 3:4563f5d3:::rbd_data.1136483b7e16.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 67.001189010000004, 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.039714705000000003, 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185252 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.608 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490857+0000", 2024-09-16T14:10:22.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490859+0000", 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.08e-06 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490869+0000", 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490893+0000", 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4176e-05 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490905+0000", 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1544e-05 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.490986+0000", 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1248999999999994e-05 2024-09-16T14:10:22.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.491117+0000", 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013125100000000001 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.502244+0000", 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0111267 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:22.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530272+0000", 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028028259999999999 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530393+0000", 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121125 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530506+0000", 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112471 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:15.530571+0000", 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5742999999999997e-05 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:194522 3.6 3:67cae2ed:::rbd_mirroring:head [call rbd.mirror_image_get in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.604752722999997, 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.02278196, 2024-09-16T14:10:22.617 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 194522 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887293+0000", 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887296+0000", 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1e-06 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.887310+0000", 2024-09-16T14:10:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3406e-05 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909564+0000", 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022254260000000001 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909597+0000", 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2869e-05 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.909755+0000", 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158426 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:33.910075+0000", 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031989900000000001 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:22.621 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:22.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:22.623 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:22.620+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:22.724 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:22.718+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:22.731 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:22.731 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:22.731 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:22.731 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:22.741 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:22.824 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:22.820+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:22.845 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:22.845 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:22.846 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:22.846 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:22.846 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:22.855 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:22.925 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:22.920+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139577 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.173866605, 2024-09-16T14:10:23.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23304406499999999, 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139577 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830188+0000", 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9400000000000001e-07 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830191+0000", 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.531e-06 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830197+0000", 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6289999999999998e-06 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830200+0000", 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.602e-06 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054193+0000", 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22399268899999999 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.054314+0000", 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121192 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063089+0000", 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008775082 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063153+0000", 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3743000000000003e-05 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063232+0000", 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9202999999999994e-05 2024-09-16T14:10:23.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139578 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.173850266, 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23316504599999999, 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139578 2024-09-16T14:10:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:23.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830204+0000", 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830205+0000", 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830208+0000", 2024-09-16T14:10:23.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7949999999999999e-06 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830221+0000", 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830225+0000", 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9899999999999999e-06 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055811+0000", 2024-09-16T14:10:23.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22558629699999999 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.055913+0000", 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010204699999999999 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063247+0000", 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073334029999999996 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063292+0000", 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5067000000000001e-05 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063369+0000", 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7569000000000002e-05 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139579 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:23.024 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.17382510100001, 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23327536500000001, 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139579 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:23.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.830230+0000", 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831088+0000", 2024-09-16T14:10:23.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085873899999999997 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831101+0000", 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2683999999999999e-05 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831111+0000", 2024-09-16T14:10:23.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8579999999999995e-06 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831117+0000", 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.489e-06 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.056930+0000", 2024-09-16T14:10:23.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22581299199999999 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.057005+0000", 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4459e-05 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063383+0000", 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063783249999999998 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063430+0000", 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7373999999999997e-05 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063505+0000", 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4444999999999996e-05 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139581 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.17292113400001, 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23513112999999999, 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139581 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831134+0000", 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831135+0000", 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.595e-06 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831139+0000", 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1839999999999999e-06 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831141+0000", 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2879999999999999e-06 2024-09-16T14:10:23.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.058920+0000", 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22777903099999999 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.059014+0000", 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4357999999999993e-05 2024-09-16T14:10:23.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066126+0000", 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071118680000000004 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066184+0000", 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7790000000000001e-05 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066265+0000", 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0696000000000001e-05 2024-09-16T14:10:23.035 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139582 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.172911795, 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23526259399999999, 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139582 2024-09-16T14:10:23.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831143+0000", 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831397+0000", 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025410999999999999 2024-09-16T14:10:23.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831402+0000", 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.78e-06 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831414+0000", 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2155e-05 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831426+0000", 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1698e-05 2024-09-16T14:10:23.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060575+0000", 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22914925999999999 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.060723+0000", 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014798400000000001 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066279+0000", 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055561719999999998 2024-09-16T14:10:23.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066331+0000", 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690999999999997e-05 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066405+0000", 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4744000000000001e-05 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.040 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139583 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.17262167299999, 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.24074520199999999, 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139583 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831433+0000", 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831630+0000", 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019706500000000001 2024-09-16T14:10:23.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831632+0000", 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831637+0000", 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5639999999999998e-06 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831640+0000", 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.819e-06 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.061876+0000", 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23023628099999999 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.062007+0000", 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000131132 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072049+0000", 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010041542000000001 2024-09-16T14:10:23.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072105+0000", 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6541e-05 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072178+0000", 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2867000000000006e-05 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139584 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.172402979, 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.242990489, 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139584 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.831652+0000", 2024-09-16T14:10:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838021+0000", 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063695360000000003 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838027+0000", 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838035+0000", 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1070000000000005e-06 2024-09-16T14:10:23.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838040+0000", 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5389999999999998e-06 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063710+0000", 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22566997999999999 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.063778+0000", 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7638999999999994e-05 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074551+0000", 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010772759 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074590+0000", 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9255999999999998e-05 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074642+0000", 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2351000000000001e-05 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139585 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.16600939999999, 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236677044, 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139585 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.838045+0000", 2024-09-16T14:10:23.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841910+0000", 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003864522 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841918+0000", 2024-09-16T14:10:23.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841939+0000", 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0398e-05 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841948+0000", 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8609999999999993e-06 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064824+0000", 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.222875135 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.064921+0000", 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7870999999999999e-05 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074653+0000", 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097313369999999996 2024-09-16T14:10:23.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074681+0000", 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8688e-05 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074722+0000", 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0881999999999999e-05 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139586 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.162097837, 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23284854999999999, 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139586 2024-09-16T14:10:23.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841957+0000", 2024-09-16T14:10:23.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841958+0000", 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.92e-07 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841963+0000", 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841975+0000", 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2515e-05 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841981+0000", 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7429999999999999e-06 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066461+0000", 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.224480243 2024-09-16T14:10:23.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.066532+0000", 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1060000000000001e-05 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074734+0000", 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082017050000000001 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074761+0000", 2024-09-16T14:10:23.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7423000000000002e-05 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.074805+0000", 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.409e-05 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139587 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:23.062 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.16206663200001, 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23548043199999999, 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139587 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.841988+0000", 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842772+0000", 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00078387200000000002 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842778+0000", 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842792+0000", 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4099e-05 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842799+0000", 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1210000000000001e-06 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067308+0000", 2024-09-16T14:10:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22450920599999999 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.067382+0000", 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3874000000000005e-05 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077293+0000", 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099105919999999993 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077360+0000", 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7973999999999997e-05 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077468+0000", 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107954 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139589 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:23.067 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.161247192, 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23483626699999999, 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139589 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842807+0000", 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842808+0000", 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2799999999999995e-07 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842821+0000", 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2325e-05 2024-09-16T14:10:23.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842836+0000", 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5954e-05 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842846+0000", 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9559999999999992e-06 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068348+0000", 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22550125900000001 2024-09-16T14:10:23.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.068434+0000", 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6782999999999999e-05 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077481+0000", 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090461510000000005 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077541+0000", 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0136000000000002e-05 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.077644+0000", 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102975 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139590 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.161202383, 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23745713700000001, 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139590 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.842852+0000", 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845823+0000", 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029703920000000001 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845830+0000", 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4599999999999997e-06 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845844+0000", 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3592e-05 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845851+0000", 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6729999999999992e-06 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069455+0000", 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22360397300000001 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.069555+0000", 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9164000000000001e-05 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080120+0000", 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010565747 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080204+0000", 2024-09-16T14:10:23.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3295999999999997e-05 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080309+0000", 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010584 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139591 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.158193129, 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23467278599999999, 2024-09-16T14:10:23.077 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139591 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845862+0000", 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7199999999999998e-07 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845866+0000", 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1699999999999999e-06 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845872+0000", 2024-09-16T14:10:23.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.908e-06 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845875+0000", 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.005e-06 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070862+0000", 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22498652299999999 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.070965+0000", 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103399 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080330+0000", 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093645329999999995 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080393+0000", 2024-09-16T14:10:23.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3208999999999997e-05 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080534+0000", 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014136699999999999 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.082 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139592 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.158176169, 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.234841987, 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139592 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845878+0000", 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845879+0000", 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2000000000000001e-07 2024-09-16T14:10:23.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845881+0000", 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9920000000000002e-06 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845885+0000", 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.25e-06 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845887+0000", 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4559999999999999e-06 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072237+0000", 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22634923100000001 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.072337+0000", 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9977999999999997e-05 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080546+0000", 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082090930000000006 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080609+0000", 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2967999999999997e-05 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080720+0000", 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111699 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.087 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139593 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.158164673, 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.2350189, 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139593 2024-09-16T14:10:23.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.845890+0000", 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850560+0000", 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046703580000000003 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850567+0000", 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8530000000000003e-06 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850579+0000", 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1454000000000001e-05 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850592+0000", 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3594999999999999e-05 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073701+0000", 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22310882000000001 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.073772+0000", 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0986000000000001e-05 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080736+0000", 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069637149999999997 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080803+0000", 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7508999999999993e-05 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.080909+0000", 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010561 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139595 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.15341776299999, 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23364501300000001, 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.093 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139595 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.850637+0000", 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854412+0000", 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037748410000000001 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854417+0000", 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7239999999999997e-06 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854427+0000", 2024-09-16T14:10:23.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.854432+0000", 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8679999999999998e-06 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076048+0000", 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22161597499999999 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.076162+0000", 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114821 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084092+0000", 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079291799999999992 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084162+0000", 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0316999999999998e-05 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.084282+0000", 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011995 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139601 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.145785132, 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23623802799999999, 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139601 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858270+0000", 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5200000000000002e-07 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858272+0000", 2024-09-16T14:10:23.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1619999999999998e-06 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858276+0000", 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858279+0000", 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.633e-06 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086101+0000", 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22782185599999999 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.086268+0000", 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016718599999999999 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094434+0000", 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081661159999999993 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094466+0000", 2024-09-16T14:10:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2218999999999998e-05 2024-09-16T14:10:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094508+0000", 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1226999999999997e-05 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139602 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:23.104 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.145772925, 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.236292153, 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139602 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.858282+0000", 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860498+0000", 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022163830000000002 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860505+0000", 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2099999999999996e-06 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860522+0000", 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7113999999999999e-05 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860529+0000", 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7959999999999998e-06 2024-09-16T14:10:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087795+0000", 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22726529200000001 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.087905+0000", 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110707 2024-09-16T14:10:23.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094517+0000", 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066117540000000001 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094541+0000", 2024-09-16T14:10:23.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3553999999999998e-05 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.094574+0000", 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3343000000000002e-05 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139603 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.110 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.14351800999999, 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.235214594, 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139603 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860537+0000", 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860713+0000", 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017653699999999999 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860718+0000", 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.933e-06 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860732+0000", 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3672000000000001e-05 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860738+0000", 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4690000000000002e-06 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089091+0000", 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.228352952 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.089182+0000", 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0694999999999998e-05 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095649+0000", 2024-09-16T14:10:23.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064666230000000003 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095680+0000", 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1523999999999998e-05 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095751+0000", 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1188999999999999e-05 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139604 3.17 3:ee6b4114:::rbd_object_map.11363617f81b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.143293759, 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.23506221999999999, 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139604 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.116 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.860761+0000", 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865955+0000", 2024-09-16T14:10:23.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051944649999999997 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865963+0000", 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0669999999999992e-06 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865979+0000", 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.865988+0000", 2024-09-16T14:10:23.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090692+0000", 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.22470385400000001 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.090787+0000", 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5092000000000004e-05 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095761+0000", 2024-09-16T14:10:23.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049738580000000003 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095784+0000", 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3292e-05 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:19.095823+0000", 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8702999999999999e-05 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.120 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:23.122 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:23.123 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:23.118+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:23.223 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:23.218+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:23.238 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:23.238 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:23.238 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:23.238 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:23.248 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:23.324 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:23.320+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:23.351 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:23.351 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:23.351 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:23.360 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:23.408 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2024-09-16T14:10:23.424 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:23.418+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:23.466 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:23.466 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:23.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:23.466 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:23.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.466 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123896 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6782976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.466 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:23.466 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.08866774699999, 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070616741999999996, 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123896 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.374452+0000", 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378050+0000", 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035980090000000001 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378056+0000", 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.404e-06 2024-09-16T14:10:23.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378070+0000", 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3305000000000001e-05 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378077+0000", 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444845+0000", 2024-09-16T14:10:23.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066767993999999997 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.444945+0000", 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.445069+0000", 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012377300000000001 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123911 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 10469376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.08473632900001, 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070660139999999996, 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123911 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378383+0000", 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378384+0000", 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378386+0000", 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9999999999999999e-06 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378394+0000", 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1829999999999994e-06 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378398+0000", 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448824+0000", 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070426107000000002 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.448920+0000", 2024-09-16T14:10:23.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6558000000000006e-05 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449043+0000", 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123177 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123912 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.084712137, 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070898066999999995, 2024-09-16T14:10:23.475 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123912 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.378408+0000", 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381848+0000", 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003440682 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381856+0000", 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3479999999999998e-06 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381866+0000", 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.381872+0000", 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6499999999999999e-06 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449080+0000", 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.067207046000000006 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449182+0000", 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102196 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.449306+0000", 2024-09-16T14:10:23.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012386600000000001 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123925 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 13418496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.080992866, 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070657760999999999, 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.480 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123925 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.382127+0000", 2024-09-16T14:10:23.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385870+0000", 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003743251 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385875+0000", 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4319999999999998e-06 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385886+0000", 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0424999999999999e-05 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.385890+0000", 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452574+0000", 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066683359999999997 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452666+0000", 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1842000000000006e-05 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.452785+0000", 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118873 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123956 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 3604480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.07707610099999, 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070646140999999996, 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123956 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.485 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.386044+0000", 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389886+0000", 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038425400000000002 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389893+0000", 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6080000000000001e-06 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389908+0000", 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5656999999999999e-05 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.389916+0000", 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7610000000000007e-06 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456527+0000", 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.066610931999999998 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456592+0000", 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5400000000000004e-05 2024-09-16T14:10:23.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.456690+0000", 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7243000000000004e-05 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123969 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 5570560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.07269382199999, 2024-09-16T14:10:23.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070612727, 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123969 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390426+0000", 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5699999999999998e-07 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390437+0000", 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0383999999999999e-05 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390443+0000", 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0120000000000002e-06 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390446+0000", 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.641e-06 2024-09-16T14:10:23.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460459+0000", 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070013041999999998 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.460683+0000", 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000224198 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461039+0000", 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035509300000000001 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123970 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6307840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.07266998099999, 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070726837000000001, 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.494 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123970 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390450+0000", 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3500000000000002e-07 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390452+0000", 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.716e-06 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390457+0000", 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1989999999999997e-06 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390461+0000", 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.459e-06 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461059+0000", 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070598093000000001 2024-09-16T14:10:23.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461113+0000", 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3758000000000003e-05 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461177+0000", 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3276999999999997e-05 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123971 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6553600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.072654793, 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071319536000000003, 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123971 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390465+0000", 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390467+0000", 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.978e-06 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390472+0000", 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1780000000000002e-06 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390476+0000", 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6899999999999998e-06 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461249+0000", 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070772825999999997 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461469+0000", 2024-09-16T14:10:23.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022043799999999999 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461784+0000", 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031507300000000002 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123972 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 6799360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:23.503 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.07264020700001, 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071496552000000005, 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123972 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:23.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390479+0000", 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390480+0000", 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3599999999999999e-07 2024-09-16T14:10:23.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390482+0000", 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7379999999999999e-06 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390487+0000", 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9989999999999999e-06 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390490+0000", 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6200000000000001e-06 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461802+0000", 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071312189999999998 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461873+0000", 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0963999999999993e-05 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.461976+0000", 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102705 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123973 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.07253655700001, 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071726447999999998, 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123973 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390583+0000", 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390584+0000", 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5899999999999995e-07 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390589+0000", 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7249999999999997e-06 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390600+0000", 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1486e-05 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390606+0000", 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5589999999999996e-06 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462033+0000", 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071427495999999993 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462159+0000", 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125609 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462310+0000", 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015081400000000001 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123974 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 7782400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.07250883099999, 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.071856779999999995, 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123974 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.390611+0000", 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393133+0000", 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025226070000000001 2024-09-16T14:10:23.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393139+0000", 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393168+0000", 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9675e-05 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393177+0000", 2024-09-16T14:10:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6030000000000005e-06 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462327+0000", 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.069149861000000007 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462376+0000", 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9345000000000003e-05 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.462468+0000", 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439000000000003e-05 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123982 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.06980850100001, 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070640819999999993, 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123982 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393311+0000", 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393312+0000", 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.65e-07 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393313+0000", 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6840000000000001e-06 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393342+0000", 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8772000000000002e-05 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393346+0000", 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4800000000000003e-06 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463851+0000", 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070504861000000002 2024-09-16T14:10:23.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463894+0000", 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2624999999999999e-05 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463952+0000", 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8032999999999998e-05 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123983 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 8273920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.069768334, 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070754062000000006, 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123983 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393351+0000", 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393424+0000", 2024-09-16T14:10:23.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2675000000000005e-05 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393427+0000", 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8279999999999999e-06 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393433+0000", 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3980000000000004e-06 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393437+0000", 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8469999999999998e-06 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.463983+0000", 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070545864 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464038+0000", 2024-09-16T14:10:23.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4832e-05 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464105+0000", 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7618000000000002e-05 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:123984 3.12 3:4f16eaa7:::rbd_data.11361565a6a9.0000000000000000:19 [read 9256960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.526 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.069678619, 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070821925999999993, 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 123984 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393441+0000", 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393830+0000", 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038931399999999998 2024-09-16T14:10:23.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393834+0000", 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.18e-06 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393840+0000", 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.878e-06 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.393844+0000", 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0670000000000002e-06 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464136+0000", 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070291396000000006 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464185+0000", 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9286000000000002e-05 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.464263+0000", 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7805000000000005e-05 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.531 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124041 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.06107226099999, 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.074201117999999996, 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124041 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.532 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402047+0000", 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.402048+0000", 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7000000000000005e-07 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404178+0000", 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021301850000000002 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404194+0000", 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5967000000000001e-05 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404203+0000", 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2809999999999997e-06 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469867+0000", 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.065663465000000004 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.469977+0000", 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110401 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476120+0000", 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061430699999999996 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476168+0000", 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8058e-05 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476249+0000", 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0121000000000001e-05 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.537 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:124042 3.12 3:4fb16458:::rbd_data.11368cf3b67c.0000000000000001:head [truncate 1916928] snapc 1b=[1b] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 135.058903969, 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.072485819000000007, 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 124042 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404216+0000", 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404217+0000", 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3489999999999999e-06 2024-09-16T14:10:23.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404221+0000", 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404230+0000", 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9910000000000008e-06 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.404239+0000", 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474347+0000", 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.070107866000000005 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.474431+0000", 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4171000000000004e-05 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476600+0000", 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021687619999999999 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476647+0000", 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6896e-05 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:08.476701+0000", 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4907999999999997e-05 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:182471 3.12 3:4dfc24b7:::rbd_data.11366c63f9a8.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 1743945] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 83.909342507999995, 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81434562300000002, 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 182471 2024-09-16T14:10:23.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553777+0000", 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553779+0000", 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6530000000000001e-06 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553787+0000", 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4710000000000007e-06 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553813+0000", 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5738e-05 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553821+0000", 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2859999999999999e-06 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553911+0000", 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9252999999999994e-05 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:59.553926+0000", 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5659e-05 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364578+0000", 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81065220199999999 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.364905+0000", 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032611400000000002 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.365142+0000", 2024-09-16T14:10:23.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000237073 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367610+0000", 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024686420000000001 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.367938+0000", 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032776300000000001 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368039+0000", 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100738 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:00.368123+0000", 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4030999999999996e-05 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.549 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185769 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [watch reconnect cookie 140383155395456 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.779333280000003, 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.013377462, 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185769 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683786+0000", 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683789+0000", 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7970000000000001e-06 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683800+0000", 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1253999999999999e-05 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683838+0000", 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7100000000000001e-05 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683852+0000", 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4039000000000001e-05 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683975+0000", 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123115 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op must wait for map", 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.683989+0000", 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4613999999999999e-05 2024-09-16T14:10:23.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691771+0000", 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077819619999999999 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.691787+0000", 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6053999999999999e-05 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.694814+0000", 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003026674 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.695138+0000", 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032383800000000001 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697164+0000", 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020260149999999999 2024-09-16T14:10:23.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:185770 3.b 3:d6de03a4:::rbd_header.1136d3e80e4b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.773470145999994, 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.007999884, 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 185770 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689649+0000", 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689652+0000", 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9969999999999999e-06 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689665+0000", 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2023e-05 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689688+0000", 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3045e-05 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.689703+0000", 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5387999999999999e-05 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692362+0000", 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026591169999999999 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:18.692372+0000", 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0236e-05 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696135+0000", 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.003762636 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.696215+0000", 2024-09-16T14:10:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0119999999999999e-05 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697528+0000", 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001312887 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697582+0000", 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4409e-05 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:09:19.697649+0000", 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7026000000000005e-05 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:201111 3.12 3:4da97782:::rbd_header.11369f5c332a:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e172)", 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.154295277999999, 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.81574065299999998, 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.562 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 201111 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308824+0000", 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308827+0000", 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8559999999999998e-06 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308839+0000", 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2170999999999999e-05 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308872+0000", 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3034999999999998e-05 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308888+0000", 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5158e-05 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.308987+0000", 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8924000000000004e-05 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:04.309001+0000", 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4504e-05 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.123915+0000", 2024-09-16T14:10:23.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.814914052 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.124090+0000", 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017510599999999999 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:05.124565+0000", 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047484699999999998 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.567 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.568 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.568 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:23.569 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:23.562+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:23.570 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:23.670 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:23.664+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGRecoveryDelete(3.6 e192,190 3:62b73fc6:::rbd_data.11362441f8a0.0000000000000032:head,79'505)", 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:10:23.676701+0000", 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.000163499, 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018552600000000001, 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.678 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676701+0000", 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676701+0000", 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676702+0000", 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5000000000000001e-07 2024-09-16T14:10:23.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676708+0000", 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9429999999999997e-06 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676725+0000", 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6881999999999999e-05 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676731+0000", 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5819999999999998e-06 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676796+0000", 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4966000000000001e-05 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676806+0000", 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8239999999999995e-06 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:10:23.676886+0000", 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0378999999999997e-05 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T14:10:23.682 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:23.687 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:23.771 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:23.766+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:23.842 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:23.842 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:23.843 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:23.843 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:23.852 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:23.872 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:23.868+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:23.957 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:23.957 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T14:10:23.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T14:10:23.957 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T14:10:23.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:3993 3.8 3:17e81ee6:::rbd_id.image_client.0-clone4:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 182.54303862200001, 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94742019899999996, 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3993 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409936+0000", 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409939+0000", 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0429999999999999e-06 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409953+0000", 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3339999999999999e-05 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.409990+0000", 2024-09-16T14:10:23.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7629000000000002e-05 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410003+0000", 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3115999999999999e-05 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410165+0000", 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016203300000000001 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T14:10:23.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:21.410205+0000", 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9731000000000003e-05 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.351883+0000", 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.94167789099999999 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352045+0000", 2024-09-16T14:10:23.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016199300000000001 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.352194+0000", 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014934400000000001 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.356293+0000", 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040984910000000001 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357253+0000", 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095975699999999999 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357310+0000", 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6807e-05 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:22.357357+0000", 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7024000000000002e-05 2024-09-16T14:10:23.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:37252 3.8 3:1118ba66:::rbd_data.11361fb616f9.0000000000000013:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 154.70507629900001, 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.040265510999999997, 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.965 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 37252 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247899+0000", 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247901+0000", 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247914+0000", 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2234e-05 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247952+0000", 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8279999999999999e-05 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.247968+0000", 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6356000000000001e-05 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248083+0000", 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001143 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.248223+0000", 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014080000000000001 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.259131+0000", 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010907313 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288032+0000", 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028901456999999998 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288046+0000", 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3723000000000001e-05 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288111+0000", 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5469999999999995e-05 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:07:49.288164+0000", 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2738000000000002e-05 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138680 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3047424~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.970 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.177837311, 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033887832999999999, 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138680 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775138+0000", 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.53e-07 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775140+0000", 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1500000000000002e-06 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775146+0000", 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8080000000000001e-06 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775150+0000", 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.658e-06 2024-09-16T14:10:23.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808935+0000", 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033785336999999999 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.808974+0000", 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809025+0000", 2024-09-16T14:10:23.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1190999999999998e-05 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138682 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2293760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.17781408, 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033995583000000003, 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138682 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775161+0000", 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775162+0000", 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7800000000000001e-07 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775169+0000", 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.311e-06 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775176+0000", 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0129999999999996e-06 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775180+0000", 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7969999999999998e-06 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809056+0000", 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033875893999999997 2024-09-16T14:10:23.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809100+0000", 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3992999999999997e-05 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809156+0000", 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6897000000000001e-05 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.979 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138683 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.177791133, 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034110511000000003, 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138683 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:23.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775184+0000", 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775186+0000", 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0109999999999999e-06 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775192+0000", 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3879999999999999e-06 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775196+0000", 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4050000000000002e-06 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809198+0000", 2024-09-16T14:10:23.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034001770000000001 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809242+0000", 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3778e-05 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809294+0000", 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2599999999999998e-05 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138684 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.17777462399999, 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034232166000000001, 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138684 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775200+0000", 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775201+0000", 2024-09-16T14:10:23.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775204+0000", 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775210+0000", 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.156e-06 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775213+0000", 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.811e-06 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809314+0000", 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034100244000000002 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809363+0000", 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9531999999999999e-05 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809432+0000", 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9225000000000002e-05 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138685 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.17775699800001, 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034454191000000002, 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.988 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138685 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775218+0000", 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.96e-07 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775220+0000", 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9470000000000002e-06 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775226+0000", 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8980000000000001e-06 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.775230+0000", 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7890000000000001e-06 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809483+0000", 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034252691000000002 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809566+0000", 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3324000000000004e-05 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.809672+0000", 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000106146 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138975 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2555904~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.992 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.171841254, 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033927766999999998, 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138975 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781134+0000", 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5300000000000004e-07 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781137+0000", 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3420000000000002e-06 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781143+0000", 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4810000000000003e-06 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781147+0000", 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8759999999999998e-06 2024-09-16T14:10:23.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814861+0000", 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033714064000000002 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.814945+0000", 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4209999999999995e-05 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:23.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815061+0000", 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116241 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138977 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 2818048~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.17182384900001, 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034175442, 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:23.997 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138977 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781151+0000", 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781152+0000", 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781154+0000", 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1509999999999998e-06 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781159+0000", 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6189999999999999e-06 2024-09-16T14:10:23.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781162+0000", 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0850000000000001e-06 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815087+0000", 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033924952000000001 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815192+0000", 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105035 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815327+0000", 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000134115 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:138979 3.b 3:d6dbc1c9:::rbd_data.113662e21c28.0000000000000002:1e [read 3080192~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.171469483, 2024-09-16T14:10:24.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033982361000000003, 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138979 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781505+0000", 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781506+0000", 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5799999999999998e-07 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781511+0000", 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1579999999999998e-06 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781521+0000", 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0231000000000001e-05 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.781526+0000", 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.271e-06 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815350+0000", 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033824052 2024-09-16T14:10:24.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815399+0000", 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8893000000000001e-05 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.815488+0000", 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8797999999999999e-05 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139076 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3473408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.168325624, 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033985627999999997, 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139076 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784649+0000", 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784650+0000", 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8799999999999998e-07 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784652+0000", 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.013e-06 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784658+0000", 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8019999999999997e-06 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784661+0000", 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4199999999999999e-06 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818433+0000", 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033772304000000003 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818533+0000", 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9961000000000001e-05 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818635+0000", 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010174 2024-09-16T14:10:24.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139077 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.168310338, 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034199510000000002, 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.010 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139077 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784665+0000", 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784667+0000", 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784673+0000", 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784676+0000", 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3720000000000001e-06 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818672+0000", 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033996418 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818781+0000", 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010897 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818864+0000", 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2658999999999996e-05 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139078 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.168293929, 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034403659000000003, 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.014 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139078 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784681+0000", 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-07 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784684+0000", 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0600000000000002e-06 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784689+0000", 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7869999999999998e-06 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784693+0000", 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5090000000000002e-06 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.818902+0000", 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034209451000000002 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819000+0000", 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7287999999999994e-05 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819085+0000", 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5141000000000004e-05 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139085 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:24.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.168218732, 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034564613000000001, 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139085 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784756+0000", 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784757+0000", 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1200000000000001e-06 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784763+0000", 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7300000000000002e-06 2024-09-16T14:10:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784777+0000", 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3528e-05 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784784+0000", 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819122+0000", 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034338502 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819229+0000", 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010668200000000001 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819321+0000", 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2106000000000002e-05 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139086 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.168121185, 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034678821999999998, 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139086 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.023 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784854+0000", 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784855+0000", 2024-09-16T14:10:24.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9700000000000005e-07 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784859+0000", 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0430000000000001e-06 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784868+0000", 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6630000000000005e-06 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784873+0000", 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0520000000000001e-06 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819353+0000", 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034479322999999999 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819450+0000", 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7305000000000004e-05 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819533+0000", 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2539000000000004e-05 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139087 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3768320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.168095944, 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034880802000000002, 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.027 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139087 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:24.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784879+0000", 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784882+0000", 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2670000000000001e-06 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784899+0000", 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7112000000000002e-05 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784903+0000", 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2039999999999999e-06 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819570+0000", 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034667018000000001 2024-09-16T14:10:24.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819677+0000", 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107197 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819760+0000", 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2614000000000006e-05 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139088 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.16806689000001, 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.035074464999999999, 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139088 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784908+0000", 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784909+0000", 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2300000000000001e-07 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784911+0000", 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6290000000000002e-06 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784934+0000", 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3122999999999999e-05 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.784940+0000", 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7830000000000004e-06 2024-09-16T14:10:24.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819797+0000", 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034857285000000002 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819896+0000", 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8331000000000004e-05 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.819983+0000", 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6791000000000003e-05 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139115 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3571712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.164802901, 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033901753, 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139115 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788172+0000", 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788173+0000", 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5400000000000002e-07 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788175+0000", 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3240000000000001e-06 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788182+0000", 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8859999999999998e-06 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788186+0000", 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1439999999999996e-06 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821869+0000", 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033683144999999998 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.821978+0000", 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109356 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822074+0000", 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5444000000000003e-05 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139116 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 3833856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.1647847, 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034092753000000003, 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139116 2024-09-16T14:10:24.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788190+0000", 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788191+0000", 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6100000000000001e-07 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788192+0000", 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.567e-06 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788220+0000", 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7815999999999999e-05 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788224+0000", 2024-09-16T14:10:24.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.053e-06 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822107+0000", 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.033883199000000003 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822198+0000", 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0123000000000004e-05 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822283+0000", 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5433999999999996e-05 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4406.0:139117 3.b 3:d35f6d42:::rbd_data.113662e21c28.0000000000000006:1e [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.164745522, 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034540816000000002, 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T14:10:24.045 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4406", 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139117 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788229+0000", 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788230+0000", 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788233+0000", 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5840000000000002e-06 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788239+0000", 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2269999999999998e-06 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.788243+0000", 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822555+0000", 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.034312032999999999 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822650+0000", 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5507000000000006e-05 2024-09-16T14:10:24.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T14:08:18.822770+0000", 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119862 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T14:10:24.049 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:24.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:24.051 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:24.046+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:24.152 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:24.146+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:24.160 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:24.160 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:24.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:24.253 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:24.248+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:24.271 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:24.271 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:24.272 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:24.272 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:24.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:24.354 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:24.350+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:24.386 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:10:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:24.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.145654011, 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.391 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785491969000006, 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:10:24.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:24.396 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785483618000001, 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:10:24.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:10:24.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785450933999996, 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:24.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:10:24.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:10:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785441696999996, 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.405 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:10:24.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:24.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785433165000001, 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:10:24.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785424587999998, 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:24.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:10:24.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785415227000001, 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.418 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:24.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:24.422 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785407114999998, 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:24.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:24.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:10:24.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:10:24.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785395412, 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.429 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:24.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:10:24.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785386798999994, 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.434 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:10:24.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.439 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:24.440 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785378692999998, 2024-09-16T14:10:24.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:10:24.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.440 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.440 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:10:24.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:10:24.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785370350999997, 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:10:24.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:10:24.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:10:24.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:10:24.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.450 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785361812999994, 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:10:24.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:10:24.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:24.458 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785353481000001, 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:10:24.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:10:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:10:24.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:10:24.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785345238999994, 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:10:24.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785335676000003, 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:10:24.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:10:24.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785326295000004, 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.475 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:10:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:24.480 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 93.785316653999999, 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:10:24.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:10:24.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:10:24.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:10:24.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:10:24.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:10:24.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:10:24.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:10:24.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:201147 3.1d 3:b85ce93f:::rbd_header.11367eb52311:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-16T14:10:24.486 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.225014363, 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22849376599999999, 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 201147 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:24.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157189+0000", 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.227e-06 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157199+0000", 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9469999999999998e-06 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157219+0000", 2024-09-16T14:10:24.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0035e-05 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157228+0000", 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9369999999999993e-06 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157310+0000", 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1459999999999996e-05 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157322+0000", 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2395000000000001e-05 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385317+0000", 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.227994851 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:24.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385438+0000", 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121371 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385681+0000", 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024254300000000001 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:24.492 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:24.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:10:24.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:10:24.495 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:24.488+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:24.595 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:24.590+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:24.597 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:24.598 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:24.598 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:24.598 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:24.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:10:24.617 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:10:24.640 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:24.663 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:24.679 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:24.695 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:24.690+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:24.696 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:24.713 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:24.713 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:24.715 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:24.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:10:24.734 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:24.755 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:24.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:10:24.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:24.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:24.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:24.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:24.758 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:24.759 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:24.759 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:24.814 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:24.810+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:24.832 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:24.832 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:24.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:24.898 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:24.894+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:24.943 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:24.943 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:24.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:24.999 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:24.996+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:25.096 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:25.096 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:25.099 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:25.094+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:25.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:25.200 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:25.196+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:25.204 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:25.205 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:25.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:25.302 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:25.294+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:25.315 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:25.315 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:25.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:25.403 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:25.400+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:25.415 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 4, 5, 6, 1, 3, 7] out_osds: [] dead_osds: [] live_osds: [1, 2, 4, 5, 6, 7, 3, 0] 2024-09-16T14:10:25.415 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:10:25.415 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T14:10:25.415 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-16T14:10:25.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T14:10:25.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:10:25.437 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:25.437 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:25.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:25.504 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:25.500+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:25.596 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:25.597 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:25.605 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:25.598+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:25.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:25.706 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:25.698+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:25.713 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:25.713 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:25.723 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:25.807 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:25.802+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:25.834 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:25.835 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:25.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:25.907 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:25.904+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:25.940 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:25.941 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:25.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:26.008 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:26.004+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:26.051 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:26.051 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:26.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:26.109 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:26.102+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:26.207 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:26.207 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:26.209 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:26.202+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:26.216 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:26.310 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:26.306+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:26.316 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:26.316 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:26.325 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:26.387 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T14:10:26.388 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2024-09-16T14:10:26.411 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:26.408+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:26.431 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:26.431 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:26.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:26.512 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:26.508+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:26.536 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:26.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:26.614 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:26.606+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:26.685 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:26.685 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:26.695 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:26.714 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:26.712+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:26.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:10:26.795 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:26.795 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:26.805 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:26.818 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:26.814+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:26.916 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:26.910+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:26.917 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:10:26.939 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:26.949 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:26.949 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:26.958 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:26.960 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:26.982 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:27.003 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:27.017 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:27.012+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:27.022 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:27.045 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:27.056 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:27.056 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:27.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:27.067 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:10:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:27.070 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:27.071 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:27.122 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:27.118+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:27.164 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:27.164 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:27.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:27.218 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:27.216+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:27.274 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:27.274 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:27.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:27.319 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:27.316+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:27.387 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145335 3.9 3.6ff6f89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.146743472, 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97368352000000002, 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145335 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236547+0000", 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236549+0000", 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236593+0000", 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3077000000000001e-05 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236612+0000", 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9885999999999999e-05 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236626+0000", 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3665999999999999e-05 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236726+0000", 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9452999999999998e-05 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:27.236738+0000", 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2842e-05 2024-09-16T14:10:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210223+0000", 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.97348435200000005 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:28.210231+0000", 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.2679999999999994e-06 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160914 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2740224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786581429999998, 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023851812, 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160914 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596710+0000", 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2100000000000002e-07 2024-09-16T14:10:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596711+0000", 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-06 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596714+0000", 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9019999999999999e-06 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596716+0000", 2024-09-16T14:10:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9369999999999998e-06 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620342+0000", 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023626075999999999 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620442+0000", 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010005699999999999 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620561+0000", 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119181 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160915 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2748416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786573078999993, 2024-09-16T14:10:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024051454999999999, 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160915 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596718+0000", 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0999999999999999e-07 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596720+0000", 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.248e-06 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596722+0000", 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8459999999999999e-06 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596724+0000", 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9659999999999999e-06 2024-09-16T14:10:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620642+0000", 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.023917473000000002 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620698+0000", 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6304999999999997e-05 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620769+0000", 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1206999999999998e-05 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160916 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2756608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786540395000003, 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024159905999999998, 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160916 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596751+0000", 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8500000000000002e-07 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596752+0000", 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3120000000000001e-06 2024-09-16T14:10:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596755+0000", 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.118e-06 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596758+0000", 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2819999999999999e-06 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620802+0000", 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024044680999999998 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620856+0000", 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3069000000000003e-05 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620910+0000", 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4959e-05 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160917 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2764800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786531158000003, 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024401559999999999, 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160917 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596760+0000", 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7300000000000002e-07 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596761+0000", 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1459999999999999e-06 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596764+0000", 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7539999999999998e-06 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596766+0000", 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9649999999999998e-06 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.620954+0000", 2024-09-16T14:10:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024188072000000001 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621053+0000", 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9310999999999998e-05 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621161+0000", 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107939 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160918 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2772992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786522625999993, 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024654903999999998, 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160918 2024-09-16T14:10:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596768+0000", 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596769+0000", 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.89e-07 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596770+0000", 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.161e-06 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596773+0000", 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596775+0000", 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1399999999999998e-06 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621212+0000", 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024437098000000001 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621311+0000", 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8641999999999995e-05 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621423+0000", 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0001126 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160919 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2781184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786514049000004, 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025063188, 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160919 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596777+0000", 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596778+0000", 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0899999999999999e-06 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596781+0000", 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7599999999999998e-06 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596784+0000", 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1109999999999999e-06 2024-09-16T14:10:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621623+0000", 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.024839133999999999 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621724+0000", 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010057500000000001 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621840+0000", 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011626299999999999 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160920 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2789376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786504687999994, 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025301223000000001, 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160920 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596786+0000", 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596787+0000", 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596788+0000", 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.052e-06 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596790+0000", 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9040000000000001e-06 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596792+0000", 2024-09-16T14:10:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.945e-06 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621888+0000", 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025095879000000001 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.621981+0000", 2024-09-16T14:10:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2866000000000002e-05 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622088+0000", 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106325 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160921 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2797568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786496576000005, 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025598909999999999, 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160921 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596794+0000", 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596795+0000", 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6300000000000001e-07 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596799+0000", 2024-09-16T14:10:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1130000000000003e-06 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596802+0000", 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596804+0000", 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.154e-06 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622137+0000", 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025332945999999999 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622253+0000", 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011622500000000001 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622393+0000", 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014011899999999999 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160922 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2805760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786484873000006, 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0258192, 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160922 2024-09-16T14:10:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596806+0000", 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.72e-07 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596808+0000", 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.457e-06 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596811+0000", 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8739999999999999e-06 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596813+0000", 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.886e-06 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622436+0000", 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025623468999999999 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622529+0000", 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2968999999999997e-05 2024-09-16T14:10:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622625+0000", 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6273000000000005e-05 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160923 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2813952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786476260000001, 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026054318, 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160923 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596815+0000", 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5199999999999998e-07 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596816+0000", 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1620000000000001e-06 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596819+0000", 2024-09-16T14:10:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.74e-06 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596821+0000", 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9889999999999999e-06 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622670+0000", 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.025849335000000001 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622761+0000", 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1082999999999995e-05 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622869+0000", 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010775700000000001 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160924 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2822144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786468154000005, 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026273027000000001, 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160924 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596823+0000", 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0199999999999998e-07 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596824+0000", 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2240000000000001e-06 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596827+0000", 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7190000000000001e-06 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596829+0000", 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0030000000000001e-06 2024-09-16T14:10:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622904+0000", 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026075114999999999 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.622994+0000", 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0080999999999994e-05 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623096+0000", 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010158300000000001 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160925 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2830336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786459812000004, 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026462566, 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160925 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596831+0000", 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5100000000000001e-07 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596833+0000", 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.212e-06 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596836+0000", 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9550000000000001e-06 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596838+0000", 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623123+0000", 2024-09-16T14:10:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026285307000000001 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623194+0000", 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0711999999999997e-05 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623294+0000", 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9989999999999996e-05 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160926 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2838528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786451274000001, 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026695236000000001, 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160926 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596840+0000", 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596841+0000", 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.203e-06 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596844+0000", 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596846+0000", 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0159999999999998e-06 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623327+0000", 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026481224000000001 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623421+0000", 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3468999999999996e-05 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623535+0000", 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000114209 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160927 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2846720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786442941999994, 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026918376000000001, 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160927 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596848+0000", 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5499999999999999e-07 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596849+0000", 2024-09-16T14:10:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.164e-06 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596852+0000", 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8569999999999999e-06 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596854+0000", 2024-09-16T14:10:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9099999999999999e-06 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623568+0000", 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026714094000000001 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623666+0000", 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7286000000000003e-05 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623766+0000", 2024-09-16T14:10:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010081000000000001 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160928 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2854912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786434700000001, 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027137016, 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160928 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596856+0000", 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596857+0000", 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596858+0000", 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.023e-06 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596861+0000", 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9490000000000001e-06 2024-09-16T14:10:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596863+0000", 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.091e-06 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623797+0000", 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026934177 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623892+0000", 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4740000000000004e-05 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.623993+0000", 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010177499999999999 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160929 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2863104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786425136999995, 2024-09-16T14:10:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027353180000000001, 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160929 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596866+0000", 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.65e-07 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596867+0000", 2024-09-16T14:10:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2139999999999999e-06 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596870+0000", 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596872+0000", 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8780000000000001e-06 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624028+0000", 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027156355 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624121+0000", 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2033999999999993e-05 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624219+0000", 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8498000000000005e-05 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160930 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2871296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786415755999997, 2024-09-16T14:10:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027572408, 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160930 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596875+0000", 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6100000000000002e-07 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596877+0000", 2024-09-16T14:10:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2249999999999999e-06 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596879+0000", 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7709999999999998e-06 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596881+0000", 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9810000000000002e-06 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624253+0000", 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027371360000000001 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624345+0000", 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.2207999999999995e-05 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624448+0000", 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102602 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:160931 3.14 3:28d95334:::rbd_data.1136bef3a187.0000000000000004:2a [read 2879488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 96.786406115000005, 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027785023999999998, 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 160931 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596885+0000", 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5800000000000001e-07 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596886+0000", 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.221e-06 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596889+0000", 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9689999999999999e-06 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.596891+0000", 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9139999999999998e-06 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624481+0000", 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027589354 2024-09-16T14:10:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624572+0000", 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1391999999999995e-05 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.624670+0000", 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7916000000000002e-05 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:201147 3.1d 3:b85ce93f:::rbd_header.11367eb52311:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173)", 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.226103823999999, 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.22849376599999999, 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 201147 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157187+0000", 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157189+0000", 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.227e-06 2024-09-16T14:10:27.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157199+0000", 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9469999999999998e-06 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157219+0000", 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0035e-05 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157228+0000", 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9369999999999993e-06 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157310+0000", 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1459999999999996e-05 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.157322+0000", 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2395000000000001e-05 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385317+0000", 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.227994851 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385438+0000", 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000121371 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:05.385681+0000", 2024-09-16T14:10:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024254300000000001 2024-09-16T14:10:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.483 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:27.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:10:27.485 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:27.478+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:27.585 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:27.578+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:27.636 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:27.636 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:27.636 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:27.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:10:27.685 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T14:10:27.680+0000 7f6141f94700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22354) UID: 0 2024-09-16T14:10:27.740 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:27.740 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:27.741 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:27.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:10:27.786 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:27.782+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:27.848 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.848 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:3369 3.e e185/184 3:75d984b1:::rbd_data.1136f7114f14.0000000000000111:18 v 185'454, mlcod=185'450)", 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:16.624798+0000", 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 11.220852806, 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057993660000000002, 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624798+0000", 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624798+0000", 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624799+0000", 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5199999999999996e-07 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624806+0000", 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0339999999999999e-06 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624822+0000", 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6892e-05 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624829+0000", 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1929999999999998e-06 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624879+0000", 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0185999999999997e-05 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.624904+0000", 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5134999999999999e-05 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.630549+0000", 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056451519999999996 2024-09-16T14:10:27.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:16.630597+0000", 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8022000000000002e-05 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203348 3.3 e186/185 3:c6d65799:::rbd_data.11362e0d5ebf.0000000000000016:head v 186'1312, mlcod=134'982)", 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.676222+0000", 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.169428433, 2024-09-16T14:10:27.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059874780000000001, 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676222+0000", 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676222+0000", 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676223+0000", 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.61e-07 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676229+0000", 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3709999999999996e-06 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676243+0000", 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3326000000000001e-05 2024-09-16T14:10:27.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676248+0000", 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6670000000000001e-06 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676287+0000", 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8792999999999999e-05 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.676312+0000", 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5111999999999999e-05 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.682174+0000", 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0058622199999999996 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.682210+0000", 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5327999999999998e-05 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203381 3.1b e186/185 3:db0f5094:::rbd_data.11362e0d5ebf.0000000000000037:head v 186'688, mlcod=134'457)", 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.690689+0000", 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.154961791, 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070459449999999996, 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690689+0000", 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690689+0000", 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690690+0000", 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9100000000000003e-07 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690696+0000", 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6560000000000003e-06 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690713+0000", 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6725999999999999e-05 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690720+0000", 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7510000000000002e-06 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.690993+0000", 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027328299999999998 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691020+0000", 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7299000000000001e-05 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.697683+0000", 2024-09-16T14:10:27.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066627839999999997 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.697735+0000", 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1654999999999999e-05 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203383 3.1b e186/185 3:df8db72c:::rbd_data.11362e0d5ebf.0000000000000039:head v 186'689, mlcod=134'457)", 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.691677+0000", 2024-09-16T14:10:27.862 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.153973973999999, 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060938829999999996, 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691677+0000", 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691677+0000", 2024-09-16T14:10:27.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691677+0000", 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3799999999999997e-07 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691682+0000", 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0669999999999999e-06 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691693+0000", 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0710000000000001e-05 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691697+0000", 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2300000000000002e-06 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691730+0000", 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2823000000000001e-05 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.691746+0000", 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5982999999999999e-05 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.697739+0000", 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0059933950000000003 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.697771+0000", 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1037000000000001e-05 2024-09-16T14:10:27.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(client.4406.0:203393 3.1d e186/185 3:bc6bf59c:::rbd_data.11362e0d5ebf.0000000000000043:head v 186'640, mlcod=0'0)", 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:17.699744+0000", 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.145907016000001, 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063103760000000004, 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699744+0000", 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699744+0000", 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699744+0000", 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7300000000000005e-07 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699752+0000", 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5449999999999998e-06 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699763+0000", 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1226999999999999e-05 2024-09-16T14:10:27.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699768+0000", 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0170000000000004e-06 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699818+0000", 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9452e-05 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.699838+0000", 2024-09-16T14:10:27.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0863999999999999e-05 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.705984+0000", 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061452750000000004 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:17.706054+0000", 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0222999999999995e-05 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:203588 3.3 3:c04552a4:::rbd_object_map.1136dd12cea0:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:18.510566+0000", 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 9.3350850389999991, 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0047273269999999, 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 203588 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.872 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:18.510566+0000", 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:18.510566+0000", 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:18.510568+0000", 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1210000000000001e-06 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:18.510575+0000", 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3980000000000002e-06 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:18.510589+0000", 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3579e-05 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:18.510599+0000", 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0811e-05 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:18.510664+0000", 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4233999999999995e-05 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:18.510677+0000", 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3705e-05 2024-09-16T14:10:27.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.509300+0000", 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.99862229000000002 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.509466+0000", 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016592199999999999 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.509674+0000", 2024-09-16T14:10:27.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020841999999999999 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.513651+0000", 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039773259999999998 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:27.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.515184+0000", 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001532959 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.515246+0000", 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1376000000000003e-05 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.515293+0000", 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7185999999999999e-05 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.878 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "MOSDPGRecoveryDelete(3.13 e187,185 3:cdf52371:::rbd_data.1136a808e66d.0000000000000002:1d,172'692)", 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:19.478330+0000", 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 8.3673201529999996, 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0073826020000000003, 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.478330+0000", 2024-09-16T14:10:27.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.478330+0000", 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.478332+0000", 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.668e-06 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.478345+0000", 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.238e-05 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.478376+0000", 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1316999999999998e-05 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.478390+0000", 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3961999999999999e-05 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.478480+0000", 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0468000000000002e-05 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.478494+0000", 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4056000000000001e-05 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:19.485713+0000", 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072187509999999998 2024-09-16T14:10:27.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:963 3.13 e192/189 3:ca9168fb:::rbd_data.11369af4c40c.0000000000000003:8 v 192'743, mlcod=186'737)", 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:23.419153+0000", 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 4.4264978109999999, 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0075905019999999998, 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.419153+0000", 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.419153+0000", 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.419154+0000", 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3500000000000005e-07 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.419163+0000", 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.251e-06 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.419189+0000", 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5976e-05 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.419196+0000", 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3690000000000001e-06 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.419522+0000", 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032588099999999998 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.419554+0000", 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1996999999999999e-05 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.426689+0000", 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071349400000000002 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:23.426743+0000", 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4252999999999997e-05 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:203939 3.3 3:c0e631c4:::rbd_header.1136494ba971:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-16T14:10:27.887 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.405101+0000", 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4405494120000002, 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011211459999999999, 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 203939 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.405101+0000", 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.405101+0000", 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.405104+0000", 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.3810000000000002e-06 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.405120+0000", 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6534000000000001e-05 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.405134+0000", 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4088e-05 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.405140+0000", 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2709999999999998e-06 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.405203+0000", 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2915000000000003e-05 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.405323+0000", 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119529 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.406823+0000", 2024-09-16T14:10:27.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014999239999999999 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.413005+0000", 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0061823620000000003 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.416241+0000", 2024-09-16T14:10:27.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032355790000000001 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.416286+0000", 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4994000000000003e-05 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.416313+0000", 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.6883e-05 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:970 3.13 e193/189 3:cad8e952:::rbd_data.1136494ba971.0000000000000014:7 v 193'749, mlcod=192'744)", 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.409700+0000", 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4359505650000002, 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071623879999999996, 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.409700+0000", 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.409700+0000", 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.409700+0000", 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4000000000000002e-07 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.409706+0000", 2024-09-16T14:10:27.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.1410000000000004e-06 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.409712+0000", 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.409715+0000", 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0900000000000001e-06 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.413620+0000", 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0039042550000000001 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.413642+0000", 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2745999999999999e-05 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.416832+0000", 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0031895510000000001 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.416862+0000", 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0516999999999999e-05 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:971 3.13 e193/189 3:cba516a6:::rbd_data.1136494ba971.000000000000006f:7 v 193'751, mlcod=193'747)", 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.417936+0000", 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4277141879999999, 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060907260000000003, 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.417936+0000", 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.417936+0000", 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.417937+0000", 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8100000000000001e-07 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.417944+0000", 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6869999999999997e-06 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.417956+0000", 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2208000000000001e-05 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.417962+0000", 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8969999999999996e-06 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.418001+0000", 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9347000000000001e-05 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.418021+0000", 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9755999999999999e-05 2024-09-16T14:10:27.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.423885+0000", 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.005863516 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.424027+0000", 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014243400000000001 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:203943 3.3 3:c0e631c4:::rbd_header.1136494ba971:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.423760+0000", 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.421890629, 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0065108910000000004, 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 203943 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.903 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.423760+0000", 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.423760+0000", 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.423762+0000", 2024-09-16T14:10:27.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0430000000000002e-06 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.423770+0000", 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3790000000000006e-06 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.423786+0000", 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6076999999999999e-05 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.423794+0000", 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5939999999999996e-06 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.423905+0000", 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000111074 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.424057+0000", 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152082 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.424232+0000", 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000175253 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.425601+0000", 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013682970000000001 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430119+0000", 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045181029999999999 2024-09-16T14:10:27.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430234+0000", 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011471700000000001 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430271+0000", 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7271999999999998e-05 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.4.0:2575 3.1b e193/188 3:d9580610:::rbd_data.1136494ba971.000000000000003a:7 v 193'718, mlcod=192'715)", 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.428152+0000", 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4174988969999998, 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.011134297, 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.428152+0000", 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.428152+0000", 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.428152+0000", 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1899999999999998e-07 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.428157+0000", 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.7500000000000003e-06 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.428167+0000", 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9350000000000006e-06 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.428171+0000", 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6800000000000001e-06 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430123+0000", 2024-09-16T14:10:27.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001951461 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430150+0000", 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7365999999999999e-05 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439243+0000", 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090924890000000005 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439286+0000", 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3297000000000002e-05 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.4.0:2576 3.1b e193/188 3:dd61974c:::rbd_data.1136494ba971.0000000000000074:7 v 193'720, mlcod=192'715)", 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.430014+0000", 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4156367599999999, 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094854380000000005, 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.913 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430014+0000", 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430014+0000", 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430014+0000", 2024-09-16T14:10:27.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4199999999999995e-07 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430021+0000", 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4119999999999998e-06 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430034+0000", 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3332e-05 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430040+0000", 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4029999999999998e-06 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432308+0000", 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002268702 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432328+0000", 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0020000000000001e-05 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439468+0000", 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0071398379999999999 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439499+0000", 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1089e-05 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.917 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:203945 3.3 3:c0e631c4:::rbd_header.1136494ba971:head [watch unwatch cookie 140383154110432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e193)", 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.430524+0000", 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.415126297, 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089406520000000003, 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 203945 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430524+0000", 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430524+0000", 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430525+0000", 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3099999999999997e-07 2024-09-16T14:10:27.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430528+0000", 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1999999999999999e-06 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430535+0000", 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3119999999999996e-06 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.430539+0000", 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8139999999999998e-06 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432445+0000", 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0019058 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432542+0000", 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7307999999999997e-05 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2", 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432655+0000", 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000112743 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.437285+0000", 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046294220000000002 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439298+0000", 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0020134720000000001 2024-09-16T14:10:27.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439362+0000", 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3983e-05 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439465+0000", 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010296700000000001 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.2.0:3416 3.e e193/184 3:767f2ffb:::rbd_data.1136494ba971.0000000000000155:7 v 193'483, mlcod=192'478)", 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.431974+0000", 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4136761330000001, 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0057648049999999996, 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.431974+0000", 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.431974+0000", 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.431975+0000", 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0599999999999999e-07 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.431982+0000", 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2979999999999997e-06 2024-09-16T14:10:27.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.431999+0000", 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7088e-05 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432005+0000", 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5039999999999999e-06 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432055+0000", 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9746000000000001e-05 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432090+0000", 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4884e-05 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.437702+0000", 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0056124249999999999 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.437739+0000", 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7054000000000001e-05 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:203946 3.3 3:c0e631c4:::rbd_header.1136494ba971:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.432506+0000", 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4131450569999999, 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0072844279999999999, 2024-09-16T14:10:27.928 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 203946 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432506+0000", 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432506+0000", 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432507+0000", 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.004e-06 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432512+0000", 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7370000000000003e-06 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432528+0000", 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5642e-05 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.432533+0000", 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8369999999999996e-06 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.433171+0000", 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00063835499999999996 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for rw locks", 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.433222+0000", 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0794999999999997e-05 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439576+0000", 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063544689999999997 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439698+0000", 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012180099999999999 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.439790+0000", 2024-09-16T14:10:27.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1787999999999996e-05 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.4.0:2577 3.1b e193/188 3:df00f7c2:::rbd_data.1136494ba971.0000000000000072:7 v 193'722, mlcod=193'718)", 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.440686+0000", 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4049646689999999, 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0066037889999999997, 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.440686+0000", 2024-09-16T14:10:27.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.440686+0000", 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.440687+0000", 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1500000000000003e-07 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.440693+0000", 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5760000000000002e-06 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.440716+0000", 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2667000000000002e-05 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.440723+0000", 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5769999999999999e-06 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441038+0000", 2024-09-16T14:10:27.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00031522599999999999 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441067+0000", 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8792000000000001e-05 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.447258+0000", 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.006191579 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.447290+0000", 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1557000000000003e-05 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.4.0:2579 3.1b e193/188 3:dfaae061:::rbd_data.1136494ba971.000000000000007a:7 v 193'724, mlcod=193'718)", 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.441509+0000", 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.4041415480000001, 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0087118260000000006, 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.939 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441509+0000", 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441509+0000", 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441510+0000", 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3799999999999997e-07 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441515+0000", 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.0799999999999996e-06 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441530+0000", 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5051e-05 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441534+0000", 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6829999999999999e-06 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441779+0000", 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00024487000000000001 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.441801+0000", 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1359999999999999e-05 2024-09-16T14:10:27.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.450178+0000", 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0083770609999999999 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.450221+0000", 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3183000000000003e-05 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_repop(osd.3.0:985 3.1d e193/188 3:bebddf32:::rbd_data.1136494ba971.000000000000002d:7 v 193'651, mlcod=189'647)", 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:24.448119+0000", 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.3975315290000001, 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0063745889999999999, 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.448119+0000", 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.448119+0000", 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.448120+0000", 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.083e-06 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.448129+0000", 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5669999999999995e-06 2024-09-16T14:10:27.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.448141+0000", 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1817999999999999e-05 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.448148+0000", 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9449999999999993e-06 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.448205+0000", 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6848000000000002e-05 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.448235+0000", 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.004e-05 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.454460+0000", 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0062244190000000001 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:27.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:24.454494+0000", 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3868999999999997e-05 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:27.948 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:27.949 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:27.942+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:27.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:10:28.051 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T14:10:28.048+0000 7fc4ef580700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22356) UID: 0 2024-09-16T14:10:28.103 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:28.104 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:28.104 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:28.104 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:28.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:10:28.152 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:28.148+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:28.216 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:28.217 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:28.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:10:28.252 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T14:10:28.248+0000 7f0a1c277700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22357) UID: 0 2024-09-16T14:10:28.328 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:28.328 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:28.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:28.328 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:28.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44965 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2890752~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.419980374, 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183204530000001, 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.329 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44965 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903577+0000", 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2499999999999995e-07 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903580+0000", 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.7989999999999998e-06 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903589+0000", 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903595+0000", 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8660000000000002e-06 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906837+0000", 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032425650000000002 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906842+0000", 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921784+0000", 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.014941895 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921838+0000", 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3585999999999998e-05 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921897+0000", 2024-09-16T14:10:28.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9880000000000003e-05 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44966 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2903040~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.419947433, 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0184433399999999, 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.334 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44966 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903610+0000", 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903614+0000", 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2550000000000002e-06 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903623+0000", 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3330000000000003e-06 2024-09-16T14:10:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903629+0000", 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6629999999999998e-06 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906866+0000", 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032368710000000001 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906870+0000", 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0770000000000001e-06 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921927+0000", 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0150569220000001 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.921979+0000", 2024-09-16T14:10:28.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2228000000000002e-05 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922053+0000", 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4379000000000001e-05 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44967 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2915328~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:28.339 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41992275000001, 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0185693469999999, 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44967 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:28.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903635+0000", 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2799999999999996e-07 2024-09-16T14:10:28.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903638+0000", 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.9340000000000002e-06 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903647+0000", 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.5979999999999997e-06 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903652+0000", 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906891+0000", 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032387039999999998 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906895+0000", 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5589999999999999e-06 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922082+0000", 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0151862899999999 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922145+0000", 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3647999999999998e-05 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922204+0000", 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8665000000000002e-05 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44968 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2927616~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41989952099999, 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197917599999999, 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44968 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:28.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903658+0000", 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1200000000000003e-07 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903665+0000", 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6479999999999997e-06 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903674+0000", 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0489999999999993e-06 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.903680+0000", 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6339999999999998e-06 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906917+0000", 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032370150000000002 2024-09-16T14:10:28.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.906921+0000", 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6979999999999997e-06 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.922225+0000", 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.015303729 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923163+0000", 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093733300000000005 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923450+0000", 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00028704199999999999 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44970 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 2952192~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41801069300001, 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0183644780000001, 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44970 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905547+0000", 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4399999999999999e-07 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905551+0000", 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.675e-06 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905561+0000", 2024-09-16T14:10:28.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7359999999999996e-06 2024-09-16T14:10:28.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.905567+0000", 2024-09-16T14:10:28.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.3130000000000004e-06 2024-09-16T14:10:28.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907261+0000", 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001693817 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907265+0000", 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5379999999999996e-06 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923704+0000", 2024-09-16T14:10:28.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.016438658 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923792+0000", 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8209999999999997e-05 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.923911+0000", 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000118787 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44974 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3001344~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.416551075, 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0189739520000001, 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44974 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907006+0000", 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907007+0000", 2024-09-16T14:10:28.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1100000000000001e-07 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907009+0000", 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2309999999999999e-06 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907015+0000", 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7470000000000002e-06 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907018+0000", 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5130000000000001e-06 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907359+0000", 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00034130199999999998 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907364+0000", 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0879999999999997e-06 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.924674+0000", 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0173104340000001 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925824+0000", 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011500779999999999 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.925980+0000", 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000156148 2024-09-16T14:10:28.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44975 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3013632~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41653530599999, 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.01922888, 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44975 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907022+0000", 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0699999999999998e-07 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907024+0000", 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8810000000000001e-06 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907030+0000", 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7039999999999999e-06 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907034+0000", 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9550000000000002e-06 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907383+0000", 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000349477 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907388+0000", 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.8069999999999999e-06 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926030+0000", 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0186412789999999 2024-09-16T14:10:28.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926129+0000", 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9180999999999998e-05 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926251+0000", 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012218900000000001 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44976 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3025920~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41651955399999, 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0196300330000001, 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44976 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907038+0000", 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5499999999999999e-07 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907040+0000", 2024-09-16T14:10:28.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1490000000000001e-06 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907046+0000", 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4920000000000001e-06 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907050+0000", 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7730000000000001e-06 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907409+0000", 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00035910299999999999 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907413+0000", 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.155e-06 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926276+0000", 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.018863707 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926547+0000", 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000270833 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926668+0000", 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012046599999999999 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44977 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3038208~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41650412300001, 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019786233, 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.375 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44977 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907053+0000", 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907054+0000", 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.15e-07 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907056+0000", 2024-09-16T14:10:28.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.063e-06 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907062+0000", 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8470000000000001e-06 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907065+0000", 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.664e-06 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907433+0000", 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000367797 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907437+0000", 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3189999999999996e-06 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926712+0000", 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.019275178 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926777+0000", 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4111999999999999e-05 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926839+0000", 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2837999999999997e-05 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44978 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3050496~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.416488485, 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020036438, 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.381 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44978 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907069+0000", 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6600000000000002e-07 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907071+0000", 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0099999999999998e-06 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907077+0000", 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6659999999999996e-06 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907080+0000", 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.478e-06 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907456+0000", 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000375478 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907460+0000", 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.002e-06 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926875+0000", 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0194153690000001 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.926974+0000", 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8453000000000001e-05 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927105+0000", 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013161600000000001 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44979 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3062784~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.416473346, 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0203377890000001, 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44979 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907084+0000", 2024-09-16T14:10:28.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.58e-07 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907086+0000", 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0040000000000002e-06 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907091+0000", 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.9350000000000002e-06 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907094+0000", 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1499999999999999e-06 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907478+0000", 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038376099999999999 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907482+0000", 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:28.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927212+0000", 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0197298690000001 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927315+0000", 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102404 2024-09-16T14:10:28.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927422+0000", 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000106999 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44980 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3075072~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41645850899999, 2024-09-16T14:10:28.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205273530000001, 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44980 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907099+0000", 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9200000000000002e-07 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907108+0000", 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8279999999999992e-06 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907114+0000", 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7699999999999998e-06 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907118+0000", 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7799999999999998e-06 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907501+0000", 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038294499999999999 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907504+0000", 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.8999999999999999e-06 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927453+0000", 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0199482010000001 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927527+0000", 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4697999999999995e-05 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927626+0000", 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8838999999999993e-05 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.397 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44981 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3087360~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41643550500001, 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207044919999999, 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44981 2024-09-16T14:10:28.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907122+0000", 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2800000000000002e-07 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907125+0000", 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.266e-06 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907130+0000", 2024-09-16T14:10:28.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.2249999999999999e-06 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907134+0000", 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9589999999999997e-06 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907523+0000", 2024-09-16T14:10:28.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00038892899999999999 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907527+0000", 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927654+0000", 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0201273200000001 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927735+0000", 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1590999999999999e-05 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927826+0000", 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0835999999999996e-05 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44982 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3099648~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41642028000001, 2024-09-16T14:10:28.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209414880000001, 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44982 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907137+0000", 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9999999999999998e-07 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907140+0000", 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0729999999999999e-06 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907145+0000", 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4870000000000002e-06 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907149+0000", 2024-09-16T14:10:28.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.523e-06 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907545+0000", 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00039664099999999999 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907549+0000", 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2749999999999997e-06 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927850+0000", 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.020300379 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.927952+0000", 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102117 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928079+0000", 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012659300000000001 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.408 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44983 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3111936~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41640509600001, 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0212128709999999, 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44983 2024-09-16T14:10:28.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907152+0000", 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907153+0000", 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1600000000000002e-07 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907155+0000", 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8899999999999999e-06 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907160+0000", 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6030000000000004e-06 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907164+0000", 2024-09-16T14:10:28.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4979999999999998e-06 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907569+0000", 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00040559199999999998 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907573+0000", 2024-09-16T14:10:28.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7919999999999999e-06 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928112+0000", 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0205387100000001 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928214+0000", 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000102125 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928365+0000", 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015124499999999999 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44984 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3124224~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41638985099999, 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0214236350000001, 2024-09-16T14:10:28.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44984 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907167+0000", 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907168+0000", 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.72e-07 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907170+0000", 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.018e-06 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907175+0000", 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6400000000000002e-06 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907179+0000", 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5530000000000001e-06 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907593+0000", 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041391300000000002 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907597+0000", 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.4950000000000002e-06 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928389+0000", 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0207917369999999 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928471+0000", 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1444000000000002e-05 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928591+0000", 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000120463 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:44985 3.d 3:b4b1f567:::rbd_data.113681563d6c.0000000000000000:15 [read 3136512~0] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e59)", 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.41637469299999, 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.021561706, 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 44985 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907183+0000", 2024-09-16T14:10:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.01e-07 2024-09-16T14:10:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907185+0000", 2024-09-16T14:10:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.891e-06 2024-09-16T14:10:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907191+0000", 2024-09-16T14:10:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.614e-06 2024-09-16T14:10:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907194+0000", 2024-09-16T14:10:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5559999999999999e-06 2024-09-16T14:10:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907701+0000", 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000507319 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:53.907705+0000", 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7400000000000002e-06 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928620+0000", 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0209145749999999 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928685+0000", 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.4954000000000002e-05 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:07:54.928744+0000", 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9655999999999999e-05 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200434 3.11 3:8f4127d4:::rbd_data.113619b3ee73.0000000000000003:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.225317995000001, 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0847031039999999, 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:28.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200434 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098239+0000", 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098242+0000", 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.39e-06 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098250+0000", 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0870000000000007e-06 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098270+0000", 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9760999999999999e-05 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098281+0000", 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1258e-05 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.099140+0000", 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00085896699999999998 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.099150+0000", 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0516999999999999e-05 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180747+0000", 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0815962020000001 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180857+0000", 2024-09-16T14:10:28.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011097199999999999 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 2,3", 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.180980+0000", 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012213400000000001 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.181989+0000", 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001009338 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182283+0000", 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00029419 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182697+0000", 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00041376599999999998 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182774+0000", 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6965000000000006e-05 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.182942+0000", 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016855700000000001 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200437 3.a 3:5303f819:::rbd_data.113619b3ee73.0000000000000006:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.22510166, 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0809326290000001, 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200437 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098456+0000", 2024-09-16T14:10:28.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098457+0000", 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9600000000000008e-07 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098462+0000", 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4070000000000001e-06 2024-09-16T14:10:28.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098476+0000", 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4243999999999999e-05 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098486+0000", 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.3950000000000007e-06 2024-09-16T14:10:28.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098546+0000", 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0717999999999997e-05 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098558+0000", 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1089e-05 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173098+0000", 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0745404629999999 2024-09-16T14:10:28.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173311+0000", 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021308399999999999 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.173553+0000", 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000241964 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:28.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.177396+0000", 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00384337 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179118+0000", 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017211419999999999 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179185+0000", 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7290000000000001e-05 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179388+0000", 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00020346700000000001 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:200439 3.a 3:53cece2e:::rbd_data.113619b3ee73.0000000000000008:head [delete] snapc 16=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e168)", 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.225062191999999, 2024-09-16T14:10:28.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0811117130000001, 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 200439 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098495+0000", 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098496+0000", 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.7000000000000004e-07 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098499+0000", 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.2710000000000001e-06 2024-09-16T14:10:28.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098509+0000", 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0066e-05 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098515+0000", 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098588+0000", 2024-09-16T14:10:28.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3070000000000003e-05 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:00.098596+0000", 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9880000000000004e-06 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174400+0000", 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0758035370000001 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174548+0000", 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014881900000000001 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 3", 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.174732+0000", 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018332799999999999 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.178747+0000", 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0040152809999999999 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179446+0000", 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00069927300000000004 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179505+0000", 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.8573999999999999e-05 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:10:01.179607+0000", 2024-09-16T14:10:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101951 2024-09-16T14:10:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.449 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:28.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:10:28.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:10:28.453 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:28.446+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:28.553 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T14:10:28.548+0000 7f08d0955700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22360) UID: 0 2024-09-16T14:10:28.556 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:28.556 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2024-09-16T14:10:28.556 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:28.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:10:28.654 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-09-16T14:10:28.650+0000 7fa3e8bd2700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22182) UID: 0 2024-09-16T14:10:28.663 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2024-09-16T14:10:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2024-09-16T14:10:28.663 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2024-09-16T14:10:28.663 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:28.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:10:28.755 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:28.750+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:28.775 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:28.775 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2024-09-16T14:10:28.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2024-09-16T14:10:28.775 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2024-09-16T14:10:28.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:145660 3.16 3:6bcb9af2:::rbd_header.113679f86037:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.252469589, 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84896851399999995, 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 145660 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.776 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518898+0000", 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518900+0000", 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.4530000000000001e-06 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518912+0000", 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1372999999999999e-05 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518940+0000", 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.8804000000000001e-05 2024-09-16T14:10:28.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.518951+0000", 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0647e-05 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519040+0000", 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8900000000000006e-05 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:31.519056+0000", 2024-09-16T14:10:28.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.6260999999999999e-05 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367114+0000", 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84805743 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367330+0000", 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021633899999999999 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:32.367866+0000", 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00053630699999999997 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:153623 3.8 3:12a349bd:::rbd_data.1136a570af09.0000000000000058:head [set-alloc-hint object_size 1048576 write_size 1048576,write 1022264~26312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 107.24719825, 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.85101481400000001, 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 153623 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524169+0000", 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524171+0000", 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9199999999999998e-06 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524205+0000", 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.4127e-05 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524222+0000", 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7271000000000001e-05 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524230+0000", 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5730000000000002e-06 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524332+0000", 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010185 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:41.524350+0000", 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8232000000000001e-05 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.371968+0000", 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.84761813500000005 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372094+0000", 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012619199999999999 2024-09-16T14:10:28.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 4", 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.372283+0000", 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018818 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.373337+0000", 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001053888 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375094+0000", 2024-09-16T14:10:28.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0017573269999999999 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375148+0000", 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.3773999999999997e-05 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:42.375184+0000", 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6344999999999998e-05 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161230 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 3964928~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.064689736000005, 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062348349999999997, 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161230 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.706678+0000", 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709515+0000", 2024-09-16T14:10:28.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0028373769999999999 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709522+0000", 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709537+0000", 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4856e-05 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.709544+0000", 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0999999999999998e-06 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768795+0000", 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059251052999999998 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.768903+0000", 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000108223 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.769026+0000", 2024-09-16T14:10:28.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000123118 2024-09-16T14:10:28.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.793 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:28.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:28.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161247 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4104192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.061289204999994, 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062427925000000002, 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:28.794 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161247 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.710078+0000", 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713761+0000", 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036832029999999999 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713768+0000", 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1410000000000002e-06 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713782+0000", 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.4608e-05 2024-09-16T14:10:28.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.713789+0000", 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.387e-06 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772297+0000", 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058508207999999999 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:28.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772399+0000", 2024-09-16T14:10:28.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101927 2024-09-16T14:10:28.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:28.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:28.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:28.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.772506+0000", 2024-09-16T14:10:28.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000107451 2024-09-16T14:10:28.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161329 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3989504~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.057237925999999, 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062321718999999998, 2024-09-16T14:10:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161329 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.714129+0000", 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717595+0000", 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034656230000000001 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717604+0000", 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717613+0000", 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1959999999999997e-06 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717622+0000", 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8540000000000003e-06 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776306+0000", 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058683854000000001 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776378+0000", 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.1507000000000005e-05 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.776451+0000", 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3671999999999995e-05 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161347 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4136960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.053485010000003, 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063027675000000005, 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161347 2024-09-16T14:10:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717882+0000", 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717883+0000", 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.7399999999999999e-07 2024-09-16T14:10:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717885+0000", 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9759999999999998e-06 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717895+0000", 2024-09-16T14:10:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.717899+0000", 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3089999999999997e-06 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779479+0000", 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061579580000000002 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.779550+0000", 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0919000000000003e-05 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.780910+0000", 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013604120000000001 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.052 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161370 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4050944~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.049405481999997, 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062305246000000002, 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161370 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721962+0000", 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.6199999999999999e-07 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721964+0000", 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.176e-06 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721970+0000", 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.7939999999999999e-06 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721974+0000", 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.0980000000000004e-06 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784123+0000", 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062148562999999997 2024-09-16T14:10:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784193+0000", 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0254999999999997e-05 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784267+0000", 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3998000000000005e-05 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161371 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4059136~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.049388879999995, 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062460235000000003, 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161371 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721979+0000", 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.96e-07 2024-09-16T14:10:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721981+0000", 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.0889999999999998e-06 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721987+0000", 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2920000000000001e-06 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721991+0000", 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.5429999999999998e-06 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784290+0000", 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062299186999999999 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784363+0000", 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.3347000000000001e-05 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784439+0000", 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5381000000000002e-05 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161372 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4067328~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.049372903999995, 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06261195, 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161372 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721994+0000", 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721995+0000", 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.3299999999999998e-07 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.721997+0000", 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2539999999999999e-06 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722005+0000", 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7219999999999999e-06 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722009+0000", 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.76e-06 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784461+0000", 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062452444000000003 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784532+0000", 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0706999999999999e-05 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784606+0000", 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.4729999999999998e-05 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161373 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4075520~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.049354952000002, 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062766050000000004, 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161373 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.722012+0000", 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725618+0000", 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036053970000000002 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725623+0000", 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.5289999999999999e-06 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725635+0000", 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1144e-05 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.725641+0000", 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.0349999999999998e-06 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784628+0000", 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058987462999999997 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784699+0000", 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.0903999999999997e-05 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.784779+0000", 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.9578000000000003e-05 2024-09-16T14:10:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161455 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3960832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.045159820999999, 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062339341, 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161455 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726208+0000", 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9000000000000002e-07 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726214+0000", 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.1310000000000003e-06 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726230+0000", 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.5413999999999998e-05 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726273+0000", 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.2987e-05 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788376+0000", 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062103955000000002 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788458+0000", 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.1168999999999995e-05 2024-09-16T14:10:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788547+0000", 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.9295000000000005e-05 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161456 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3969024~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.045062842999997, 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.06243021, 2024-09-16T14:10:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161456 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726305+0000", 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.61e-07 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726308+0000", 2024-09-16T14:10:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0869999999999998e-06 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726316+0000", 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726324+0000", 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.5959999999999998e-06 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788575+0000", 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062251488000000001 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788652+0000", 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6097e-05 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788735+0000", 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3175000000000002e-05 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161457 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 3977216~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.045038301999995, 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062669868000000004, 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161457 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.726329+0000", 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729579+0000", 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032499 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729585+0000", 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.575e-06 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729596+0000", 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1894e-05 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729603+0000", 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.5849999999999996e-06 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788828+0000", 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059224710999999999 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788914+0000", 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.6451000000000003e-05 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.788999+0000", 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.4752000000000004e-05 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161470 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4083712~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.041481540000007, 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062186294000000003, 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161470 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729886+0000", 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9900000000000001e-07 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729888+0000", 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1880000000000001e-06 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729895+0000", 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.674e-06 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729899+0000", 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.9249999999999997e-06 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791842+0000", 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061943338000000001 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.791946+0000", 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010320299999999999 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792072+0000", 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012656700000000001 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161471 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4091904~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.041463781999994, 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062453997999999997, 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161471 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729904+0000", 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.5499999999999998e-07 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729906+0000", 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.243e-06 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729913+0000", 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2400000000000004e-06 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729917+0000", 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.1540000000000004e-06 2024-09-16T14:10:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792101+0000", 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062184602999999998 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792208+0000", 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00010689 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792358+0000", 2024-09-16T14:10:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014941299999999999 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161472 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4100096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.041446119, 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062946515999999994, 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161472 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729921+0000", 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729922+0000", 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.3700000000000001e-07 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729924+0000", 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.2079999999999999e-06 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729931+0000", 2024-09-16T14:10:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.618e-06 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729934+0000", 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.625e-06 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792396+0000", 2024-09-16T14:10:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062461397000000002 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792766+0000", 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000370527 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792868+0000", 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000101704 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161473 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4108288~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.041428925000005, 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063123850999999995, 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161473 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.729938+0000", 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733481+0000", 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035425890000000001 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733487+0000", 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.6119999999999998e-06 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733498+0000", 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733503+0000", 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:10:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792897+0000", 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.059393525000000003 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.792974+0000", 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.7254000000000003e-05 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.793062+0000", 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8446e-05 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161483 3.16 3:6aafbb1e:::rbd_data.1136bef3a187.0000000000000005:2a [read 4190208~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.037732507000001, 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065886937000000007, 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161483 2024-09-16T14:10:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733635+0000", 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.5699999999999999e-07 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733636+0000", 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2899999999999999e-06 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733639+0000", 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.0130000000000002e-06 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.733641+0000", 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.939e-06 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799315+0000", 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.065673789999999996 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799402+0000", 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.7046999999999995e-05 2024-09-16T14:10:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.799522+0000", 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000119601 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:161582 3.16 3:6831f535:::rbd_data.1136bef3a187.0000000000000002:2a [read 4448256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.025402554999999, 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062229382, 2024-09-16T14:10:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "started", 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 161582 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.745965+0000", 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749565+0000", 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036000960000000001 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749572+0000", 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.9940000000000003e-06 2024-09-16T14:10:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749581+0000", 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.8570000000000001e-06 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.749585+0000", 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 4.6210000000000003e-06 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808048+0000", 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.058462110999999997 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808131+0000", 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 8.3706000000000001e-05 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:08:50.808194+0000", 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.2996999999999993e-05 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4406.0:185210 3.11 3:882f9b3c:::rbd_data.1136483b7e16.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 74.351890382999997, 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.986011886, 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4406", 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/99981611", 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 185210 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2024-09-16T14:10:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419477+0000", 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419479+0000", 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.669e-06 2024-09-16T14:10:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419485+0000", 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 6.224e-06 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419502+0000", 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.7071999999999998e-05 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419508+0000", 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.784e-06 2024-09-16T14:10:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419567+0000", 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 5.9265999999999998e-05 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:14.419578+0000", 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0777e-05 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2024-09-16T14:10:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.402822+0000", 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.98324447400000003 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403101+0000", 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00027876600000000001 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 1", 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.403204+0000", 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000103287 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.404297+0000", 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001092272 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405236+0000", 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093966299999999998 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405313+0000", 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 7.6877000000000003e-05 2024-09-16T14:10:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2024-09-16T14:09:15.405489+0000", 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017575499999999999 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-09-16T14:10:29.117 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:29.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:10:29.119 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:10:29.120 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:29.114+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:29.220 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:29.214+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:29.226 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:29.226 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T14:10:29.226 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:29.236 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:10:29.299 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2024-09-16T14:10:29.319 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2024-09-16T14:10:29.320 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:29.314+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:29.332 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T14:10:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T14:10:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 30, 2024-09-16T14:10:29.332 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T14:10:29.332 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T14:10:29.341 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:10:29.342 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2024-09-16T14:10:29.361 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2024-09-16T14:10:29.380 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2024-09-16T14:10:29.401 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2024-09-16T14:10:29.421 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-09-16T14:10:29.414+0000 7fe547780700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22187) UID: 0 2024-09-16T14:10:29.422 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2024-09-16T14:10:29.442 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:29.442 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:29.444 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2024-09-16T14:10:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:10:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:10:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:10:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:10:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:10:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:10:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:10:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:10:29.453 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:10:29.453 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:10:29.522 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-09-16T14:10:29.518+0000 7f3f16cf1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35922) UID: 0 2024-09-16T14:10:29.550 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:29.550 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:29.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:10:29.606 INFO:teuthology.orchestra.run.smithi079.stdout:All operations completed A-OK! 2024-09-16T14:10:29.623 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-09-16T14:10:29.618+0000 7f0dd9ed8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 47336) UID: 0 2024-09-16T14:10:29.643 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:29.643 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-16T14:10:29.652 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T14:10:29.703 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T14:10:29.704 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:29.705 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:29.705 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:10:29.810 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:29.811 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:29.819 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:10:29.919 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:29.919 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:29.928 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:10:30.022 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.022 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.031 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:10:30.125 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.125 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.134 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:10:30.233 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.233 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.242 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:10:30.343 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.344 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.352 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:10:30.456 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.457 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:10:30.572 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.572 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.581 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:10:30.684 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.684 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:10:30.794 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.794 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:10:30.900 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:30.900 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:30.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:10:31.006 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:31.006 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:31.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:10:31.115 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2024-09-16T14:10:31.115 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:10:31.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T14:10:31.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:10:31.553 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a: {} 2024-09-16T14:10:31.571 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b: {} 2024-09-16T14:10:31.589 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c: {} 2024-09-16T14:10:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:10:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:10:31.591 INFO:teuthology.orchestra.run.smithi079.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:10:31.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T14:10:32.466 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2024-09-16T14:10:32.466 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T14:10:32.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T14:10:32.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:10:33.477 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T14:10:33.477 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2024-09-16T14:10:33.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T14:10:34.480 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T14:10:34.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T14:10:35.392 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T14:10:35.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T14:10:36.491 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T14:10:36.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T14:10:37.502 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T14:10:37.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T14:10:38.517 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T14:10:38.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T14:10:39.526 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T14:10:39.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T14:10:40.529 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T14:10:40.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T14:10:41.544 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T14:10:41.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T14:10:42.559 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T14:10:42.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-16T14:10:43.580 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T14:10:43.592 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T14:10:44.580 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T14:10:44.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T14:10:45.596 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T14:10:45.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T14:10:46.600 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T14:10:46.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T14:10:47.613 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T14:10:47.625 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T14:10:47.626 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T14:10:47.626 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T14:10:47.626 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T14:10:47.626 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T14:10:47.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:47.918 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":215,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:10:47.595917+0000","last_up_change":"2024-09-16T14:10:15.439434+0000","last_in_change":"2024-09-16T14:10:20.368208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:06:58.526935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T14:07:08.375919+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":103,"snap_epoch":198,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":129,"up_from":184,"up_thru":193,"down_at":130,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":47342},{"type":"v1","addr":"172.21.15.79:6817","nonce":47342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":47342},{"type":"v1","addr":"172.21.15.79:6819","nonce":47342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":47342},{"type":"v1","addr":"172.21.15.79:6824","nonce":47342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":47342},{"type":"v1","addr":"172.21.15.79:6821","nonce":47342}]},"public_addr":"172.21.15.79:6817/47342","cluster_addr":"172.21.15.79:6819/47342","heartbeat_back_addr":"172.21.15.79:6824/47342","heartbeat_front_addr":"172.21.15.79:6821/47342","state":["exists","up"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":83,"up_thru":189,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":35927},{"type":"v1","addr":"172.21.15.79:6801","nonce":35927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":35927},{"type":"v1","addr":"172.21.15.79:6803","nonce":35927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":35927},{"type":"v1","addr":"172.21.15.79:6807","nonce":35927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":35927},{"type":"v1","addr":"172.21.15.79:6805","nonce":35927}]},"public_addr":"172.21.15.79:6801/35927","cluster_addr":"172.21.15.79:6803/35927","heartbeat_back_addr":"172.21.15.79:6807/35927","heartbeat_front_addr":"172.21.15.79:6805/35927","state":["exists","up"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":130},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":59},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:10:47.933 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T14:10:47.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:10:48.219 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:48.219 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":215,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:10:47.595917+0000","last_up_change":"2024-09-16T14:10:15.439434+0000","last_in_change":"2024-09-16T14:10:20.368208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:06:58.526935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T14:07:08.375919+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":103,"snap_epoch":198,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":129,"up_from":184,"up_thru":193,"down_at":130,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":47342},{"type":"v1","addr":"172.21.15.79:6817","nonce":47342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":47342},{"type":"v1","addr":"172.21.15.79:6819","nonce":47342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":47342},{"type":"v1","addr":"172.21.15.79:6824","nonce":47342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":47342},{"type":"v1","addr":"172.21.15.79:6821","nonce":47342}]},"public_addr":"172.21.15.79:6817/47342","cluster_addr":"172.21.15.79:6819/47342","heartbeat_back_addr":"172.21.15.79:6824/47342","heartbeat_front_addr":"172.21.15.79:6821/47342","state":["exists","up"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":83,"up_thru":189,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":35927},{"type":"v1","addr":"172.21.15.79:6801","nonce":35927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":35927},{"type":"v1","addr":"172.21.15.79:6803","nonce":35927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":35927},{"type":"v1","addr":"172.21.15.79:6807","nonce":35927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":35927},{"type":"v1","addr":"172.21.15.79:6805","nonce":35927}]},"public_addr":"172.21.15.79:6801/35927","cluster_addr":"172.21.15.79:6803/35927","heartbeat_back_addr":"172.21.15.79:6807/35927","heartbeat_front_addr":"172.21.15.79:6805/35927","state":["exists","up"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":130},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":59},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:10:48.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T14:10:48.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T14:10:48.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T14:10:48.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T14:10:48.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T14:10:48.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T14:10:48.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T14:10:48.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T14:10:48.432 INFO:teuthology.orchestra.run.smithi079.stdout:60129542196 2024-09-16T14:10:48.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T14:10:48.441 INFO:teuthology.orchestra.run.smithi079.stdout:64424509493 2024-09-16T14:10:48.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T14:10:48.446 INFO:teuthology.orchestra.run.smithi079.stdout:790273982472 2024-09-16T14:10:48.447 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T14:10:48.448 INFO:teuthology.orchestra.run.smithi079.stdout:64424509494 2024-09-16T14:10:48.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T14:10:48.450 INFO:teuthology.orchestra.run.smithi079.stdout:356482285598 2024-09-16T14:10:48.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T14:10:48.457 INFO:teuthology.orchestra.run.smithi079.stdout:64424509492 2024-09-16T14:10:48.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T14:10:48.459 INFO:teuthology.orchestra.run.smithi079.stdout:64424509492 2024-09-16T14:10:48.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T14:10:48.465 INFO:teuthology.orchestra.run.smithi079.stdout:64424509492 2024-09-16T14:10:48.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T14:10:48.899 INFO:teuthology.orchestra.run.smithi079.stdout:60129542195 2024-09-16T14:10:48.907 INFO:teuthology.orchestra.run.smithi079.stdout:64424509491 2024-09-16T14:10:48.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542196 got 60129542195 for osd.4 2024-09-16T14:10:48.918 INFO:teuthology.orchestra.run.smithi079.stdout:790273982471 2024-09-16T14:10:48.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509493 got 64424509491 for osd.5 2024-09-16T14:10:48.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 790273982472 got 790273982471 for osd.0 2024-09-16T14:10:48.944 INFO:teuthology.orchestra.run.smithi079.stdout:356482285597 2024-09-16T14:10:48.954 INFO:teuthology.orchestra.run.smithi079.stdout:64424509492 2024-09-16T14:10:48.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 356482285598 got 356482285597 for osd.3 2024-09-16T14:10:48.961 INFO:teuthology.orchestra.run.smithi079.stdout:64424509490 2024-09-16T14:10:48.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509494 got 64424509492 for osd.2 2024-09-16T14:10:48.973 INFO:teuthology.orchestra.run.smithi079.stdout:64424509490 2024-09-16T14:10:48.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509490 for osd.6 2024-09-16T14:10:48.981 INFO:teuthology.orchestra.run.smithi079.stdout:64424509490 2024-09-16T14:10:48.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509490 for osd.7 2024-09-16T14:10:48.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509490 for osd.1 2024-09-16T14:10:49.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T14:10:49.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T14:10:49.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T14:10:49.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T14:10:49.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T14:10:49.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T14:10:49.986 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T14:10:49.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T14:10:50.377 INFO:teuthology.orchestra.run.smithi079.stdout:60129542196 2024-09-16T14:10:50.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542196 got 60129542196 for osd.4 2024-09-16T14:10:50.401 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:50.429 INFO:teuthology.orchestra.run.smithi079.stdout:64424509492 2024-09-16T14:10:50.435 INFO:teuthology.orchestra.run.smithi079.stdout:64424509493 2024-09-16T14:10:50.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509492 for osd.7 2024-09-16T14:10:50.442 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:50.449 INFO:teuthology.orchestra.run.smithi079.stdout:790273982472 2024-09-16T14:10:50.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509493 got 64424509493 for osd.5 2024-09-16T14:10:50.451 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:50.457 INFO:teuthology.orchestra.run.smithi079.stdout:64424509492 2024-09-16T14:10:50.463 INFO:teuthology.orchestra.run.smithi079.stdout:64424509494 2024-09-16T14:10:50.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 790273982472 got 790273982472 for osd.0 2024-09-16T14:10:50.466 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:50.472 INFO:teuthology.orchestra.run.smithi079.stdout:356482285598 2024-09-16T14:10:50.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509492 for osd.6 2024-09-16T14:10:50.474 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:50.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509494 got 64424509494 for osd.2 2024-09-16T14:10:50.476 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:50.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 356482285598 got 356482285598 for osd.3 2024-09-16T14:10:50.485 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:50.493 INFO:teuthology.orchestra.run.smithi079.stdout:64424509492 2024-09-16T14:10:50.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509492 got 64424509492 for osd.1 2024-09-16T14:10:50.505 DEBUG:teuthology.parallel:result is None 2024-09-16T14:10:50.505 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T14:10:50.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:10:50.821 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:50.823 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:10:50.835 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-16T14:10:48.829909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":588,"num_read":281435,"num_read_kb":918233,"num_write":31188,"num_write_kb":443365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9465,"num_bytes_recovered":336866533,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27166,"ondisk_log_size":27166,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2479400,"kb_used_data":20260,"kb_used_omap":166,"kb_used_meta":2458841,"kb_avail":752495320,"statfs":{"total":773094113280,"available":770555207680,"internally_reserved":0,"allocated":20746240,"data_stored":11135051,"data_compressed":11598,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":170259,"internal_metadata":2517853933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.260246"},"pg_stats":[{"pgid":"3.1f","version":"198'1876","reported_seq":7887,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592881+0000","last_change":"2024-09-16T14:10:29.531167+0000","last_active":"2024-09-16T14:10:46.592881+0000","last_peered":"2024-09-16T14:10:46.592881+0000","last_clean":"2024-09-16T14:10:46.592881+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:10:46.592881+0000","last_undegraded":"2024-09-16T14:10:46.592881+0000","last_fullsized":"2024-09-16T14:10:46.592881+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1876,"log_dups_size":0,"ondisk_log_size":1876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:02.959438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039454019999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15599,"num_read_kb":19767,"num_write":2973,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"196'800","reported_seq":2726,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592990+0000","last_change":"2024-09-16T14:10:29.528090+0000","last_active":"2024-09-16T14:10:46.592990+0000","last_peered":"2024-09-16T14:10:46.592990+0000","last_clean":"2024-09-16T14:10:46.592990+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:10:46.592990+0000","last_undegraded":"2024-09-16T14:10:46.592990+0000","last_fullsized":"2024-09-16T14:10:46.592990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098036900000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3428,"num_read_kb":21398,"num_write":967,"num_write_kb":11613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"197'653","reported_seq":5530,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617422+0000","last_change":"2024-09-16T14:10:29.581527+0000","last_active":"2024-09-16T14:10:45.617422+0000","last_peered":"2024-09-16T14:10:45.617422+0000","last_clean":"2024-09-16T14:10:45.617422+0000","last_became_active":"2024-09-16T14:10:20.387629+0000","last_became_peered":"2024-09-16T14:10:20.387629+0000","last_unstale":"2024-09-16T14:10:45.617422+0000","last_undegraded":"2024-09-16T14:10:45.617422+0000","last_fullsized":"2024-09-16T14:10:45.617422+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:30.104851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051863780000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7537,"num_read_kb":24768,"num_write":780,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":26063912,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"198'1032","reported_seq":14527,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592725+0000","last_change":"2024-09-16T14:10:29.463373+0000","last_active":"2024-09-16T14:10:46.592725+0000","last_peered":"2024-09-16T14:10:46.592725+0000","last_clean":"2024-09-16T14:10:46.592725+0000","last_became_active":"2024-09-16T14:10:20.385216+0000","last_became_peered":"2024-09-16T14:10:20.385216+0000","last_unstale":"2024-09-16T14:10:46.592725+0000","last_undegraded":"2024-09-16T14:10:46.592725+0000","last_fullsized":"2024-09-16T14:10:46.592725+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:22.705027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013844,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":21,"num_read":15843,"num_read_kb":26936,"num_write":1369,"num_write_kb":11030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":6143753,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"197'738","reported_seq":8080,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.446977+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:20.387007+0000","last_became_peered":"2024-09-16T14:10:20.387007+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:41.660168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010460949999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10352,"num_read_kb":26550,"num_write":833,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"196'845","reported_seq":8027,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592011+0000","last_change":"2024-09-16T14:10:29.463712+0000","last_active":"2024-09-16T14:10:46.592011+0000","last_peered":"2024-09-16T14:10:46.592011+0000","last_clean":"2024-09-16T14:10:46.592011+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:10:46.592011+0000","last_undegraded":"2024-09-16T14:10:46.592011+0000","last_fullsized":"2024-09-16T14:10:46.592011+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:25.215853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017171580000000001,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":10449,"num_read_kb":28504,"num_write":803,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"197'630","reported_seq":2342,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589326+0000","last_change":"2024-09-16T14:10:29.447450+0000","last_active":"2024-09-16T14:10:45.589326+0000","last_peered":"2024-09-16T14:10:45.589326+0000","last_clean":"2024-09-16T14:10:45.589326+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:10:45.589326+0000","last_undegraded":"2024-09-16T14:10:45.589326+0000","last_fullsized":"2024-09-16T14:10:45.589326+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:41.369426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483112,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1924,"num_read_kb":28666,"num_write":709,"num_write_kb":21954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"198'968","reported_seq":6364,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592751+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592751+0000","last_peered":"2024-09-16T14:10:46.592751+0000","last_clean":"2024-09-16T14:10:46.592751+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:10:46.592751+0000","last_undegraded":"2024-09-16T14:10:46.592751+0000","last_fullsized":"2024-09-16T14:10:46.592751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050019210000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9238,"num_read_kb":21812,"num_write":1162,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"194'1351","reported_seq":6429,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593259+0000","last_change":"2024-09-16T14:10:29.531182+0000","last_active":"2024-09-16T14:10:46.593259+0000","last_peered":"2024-09-16T14:10:46.593259+0000","last_clean":"2024-09-16T14:10:46.593259+0000","last_became_active":"2024-09-16T14:10:05.123005+0000","last_became_peered":"2024-09-16T14:10:05.123005+0000","last_unstale":"2024-09-16T14:10:46.593259+0000","last_undegraded":"2024-09-16T14:10:46.593259+0000","last_fullsized":"2024-09-16T14:10:46.593259+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:44.294281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039366710000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8929,"num_read_kb":33272,"num_write":1987,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":4502377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"197'751","reported_seq":13941,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573574+0000","last_change":"2024-09-16T14:10:29.446978+0000","last_active":"2024-09-16T14:10:44.573574+0000","last_peered":"2024-09-16T14:10:44.573574+0000","last_clean":"2024-09-16T14:10:44.573574+0000","last_became_active":"2024-09-16T14:10:05.377635+0000","last_became_peered":"2024-09-16T14:10:05.377635+0000","last_unstale":"2024-09-16T14:10:44.573574+0000","last_undegraded":"2024-09-16T14:10:44.573574+0000","last_fullsized":"2024-09-16T14:10:44.573574+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:42.336862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089383699999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":14525,"num_read_kb":50269,"num_write":689,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":11596768,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"198'576","reported_seq":5188,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.571551+0000","last_change":"2024-09-16T14:10:29.448044+0000","last_active":"2024-09-16T14:10:44.571551+0000","last_peered":"2024-09-16T14:10:44.571551+0000","last_clean":"2024-09-16T14:10:44.571551+0000","last_became_active":"2024-09-16T14:10:19.498121+0000","last_became_peered":"2024-09-16T14:10:19.498121+0000","last_unstale":"2024-09-16T14:10:44.571551+0000","last_undegraded":"2024-09-16T14:10:44.571551+0000","last_fullsized":"2024-09-16T14:10:44.571551+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:08.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002716544,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4971,"num_read_kb":28564,"num_write":500,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":50024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"197'723","reported_seq":7824,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617501+0000","last_change":"2024-09-16T14:10:29.582367+0000","last_active":"2024-09-16T14:10:45.617501+0000","last_peered":"2024-09-16T14:10:45.617501+0000","last_clean":"2024-09-16T14:10:45.617501+0000","last_became_active":"2024-09-16T14:10:06.388650+0000","last_became_peered":"2024-09-16T14:10:06.388650+0000","last_unstale":"2024-09-16T14:10:45.617501+0000","last_undegraded":"2024-09-16T14:10:45.617501+0000","last_fullsized":"2024-09-16T14:10:45.617501+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:30.578665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054744670000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9236,"num_read_kb":38811,"num_write":682,"num_write_kb":10309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":10174643,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617632+0000","last_change":"2024-09-16T14:10:17.556015+0000","last_active":"2024-09-16T14:10:45.617632+0000","last_peered":"2024-09-16T14:10:45.617632+0000","last_clean":"2024-09-16T14:10:45.617632+0000","last_became_active":"2024-09-16T14:10:17.555698+0000","last_became_peered":"2024-09-16T14:10:17.555698+0000","last_unstale":"2024-09-16T14:10:45.617632+0000","last_undegraded":"2024-09-16T14:10:45.617632+0000","last_fullsized":"2024-09-16T14:10:45.617632+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:03.785460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"198'897","reported_seq":6176,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.448811+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:25.385245+0000","last_became_peered":"2024-09-16T14:10:25.385245+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:00.028708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026361219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5530,"num_read_kb":25811,"num_write":916,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":731,"num_bytes_recovered":15701912,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589505+0000","last_change":"2024-09-16T14:10:16.472462+0000","last_active":"2024-09-16T14:10:45.589505+0000","last_peered":"2024-09-16T14:10:45.589505+0000","last_clean":"2024-09-16T14:10:45.589505+0000","last_became_active":"2024-09-16T14:10:16.471977+0000","last_became_peered":"2024-09-16T14:10:16.471977+0000","last_unstale":"2024-09-16T14:10:45.589505+0000","last_undegraded":"2024-09-16T14:10:45.589505+0000","last_fullsized":"2024-09-16T14:10:45.589505+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:09.028511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"196'967","reported_seq":7724,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589578+0000","last_change":"2024-09-16T14:10:29.448302+0000","last_active":"2024-09-16T14:10:45.589578+0000","last_peered":"2024-09-16T14:10:45.589578+0000","last_clean":"2024-09-16T14:10:45.589578+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:10:45.589578+0000","last_undegraded":"2024-09-16T14:10:45.589578+0000","last_fullsized":"2024-09-16T14:10:45.589578+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:58.173875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002259535,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":10392,"num_read_kb":45651,"num_write":1244,"num_write_kb":30465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617428+0000","last_change":"2024-09-16T14:10:00.102785+0000","last_active":"2024-09-16T14:10:45.617428+0000","last_peered":"2024-09-16T14:10:45.617428+0000","last_clean":"2024-09-16T14:10:45.617428+0000","last_became_active":"2024-09-16T14:10:00.102521+0000","last_became_peered":"2024-09-16T14:10:00.102521+0000","last_unstale":"2024-09-16T14:10:45.617428+0000","last_undegraded":"2024-09-16T14:10:45.617428+0000","last_fullsized":"2024-09-16T14:10:45.617428+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:17.608389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"197'630","reported_seq":3952,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616832+0000","last_change":"2024-09-16T14:10:29.574272+0000","last_active":"2024-09-16T14:10:45.616832+0000","last_peered":"2024-09-16T14:10:45.616832+0000","last_clean":"2024-09-16T14:10:45.616832+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:10:45.616832+0000","last_undegraded":"2024-09-16T14:10:45.616832+0000","last_fullsized":"2024-09-16T14:10:45.616832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001012095,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4560,"num_read_kb":16131,"num_write":597,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593236+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:10:46.593236+0000","last_peered":"2024-09-16T14:10:46.593236+0000","last_clean":"2024-09-16T14:10:46.593236+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:10:46.593236+0000","last_undegraded":"2024-09-16T14:10:46.593236+0000","last_fullsized":"2024-09-16T14:10:46.593236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"197'780","reported_seq":9812,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573395+0000","last_change":"2024-09-16T14:10:29.447893+0000","last_active":"2024-09-16T14:10:44.573395+0000","last_peered":"2024-09-16T14:10:44.573395+0000","last_clean":"2024-09-16T14:10:44.573395+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:10:44.573395+0000","last_undegraded":"2024-09-16T14:10:44.573395+0000","last_fullsized":"2024-09-16T14:10:44.573395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064281,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":11041,"num_read_kb":12619,"num_write":857,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"198'1412","reported_seq":5232,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.571429+0000","last_change":"2024-09-16T14:10:29.449673+0000","last_active":"2024-09-16T14:10:44.571429+0000","last_peered":"2024-09-16T14:10:44.571429+0000","last_clean":"2024-09-16T14:10:44.571429+0000","last_became_active":"2024-09-16T14:10:19.505692+0000","last_became_peered":"2024-09-16T14:10:19.505692+0000","last_unstale":"2024-09-16T14:10:44.571429+0000","last_undegraded":"2024-09-16T14:10:44.571429+0000","last_fullsized":"2024-09-16T14:10:44.571429+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:54.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040141910000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":9687,"num_read_kb":19616,"num_write":1641,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617596+0000","last_change":"2024-09-16T14:10:00.110539+0000","last_active":"2024-09-16T14:10:45.617596+0000","last_peered":"2024-09-16T14:10:45.617596+0000","last_clean":"2024-09-16T14:10:45.617596+0000","last_became_active":"2024-09-16T14:10:00.104539+0000","last_became_peered":"2024-09-16T14:10:00.104539+0000","last_unstale":"2024-09-16T14:10:45.617596+0000","last_undegraded":"2024-09-16T14:10:45.617596+0000","last_fullsized":"2024-09-16T14:10:45.617596+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:07.278197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"196'1316","reported_seq":6546,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592682+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592682+0000","last_peered":"2024-09-16T14:10:46.592682+0000","last_clean":"2024-09-16T14:10:46.592682+0000","last_became_active":"2024-09-16T14:10:26.385527+0000","last_became_peered":"2024-09-16T14:10:26.385527+0000","last_unstale":"2024-09-16T14:10:46.592682+0000","last_undegraded":"2024-09-16T14:10:46.592682+0000","last_fullsized":"2024-09-16T14:10:46.592682+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1316,"log_dups_size":0,"ondisk_log_size":1316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:03.910845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051618139999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9488,"num_read_kb":24272,"num_write":1738,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":8428888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617802+0000","last_change":"2024-09-16T14:10:21.388025+0000","last_active":"2024-09-16T14:10:45.617802+0000","last_peered":"2024-09-16T14:10:45.617802+0000","last_clean":"2024-09-16T14:10:45.617802+0000","last_became_active":"2024-09-16T14:10:21.387173+0000","last_became_peered":"2024-09-16T14:10:21.387173+0000","last_unstale":"2024-09-16T14:10:45.617802+0000","last_undegraded":"2024-09-16T14:10:45.617802+0000","last_fullsized":"2024-09-16T14:10:45.617802+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:57.862024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"197'820","reported_seq":7030,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616505+0000","last_change":"2024-09-16T14:10:29.574667+0000","last_active":"2024-09-16T14:10:45.616505+0000","last_peered":"2024-09-16T14:10:45.616505+0000","last_clean":"2024-09-16T14:10:45.616505+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:10:45.616505+0000","last_undegraded":"2024-09-16T14:10:45.616505+0000","last_fullsized":"2024-09-16T14:10:45.616505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233734,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":8272,"num_read_kb":21183,"num_write":947,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617104+0000","last_change":"2024-09-16T14:10:00.104224+0000","last_active":"2024-09-16T14:10:45.617104+0000","last_peered":"2024-09-16T14:10:45.617104+0000","last_clean":"2024-09-16T14:10:45.617104+0000","last_became_active":"2024-09-16T14:10:00.104003+0000","last_became_peered":"2024-09-16T14:10:00.104003+0000","last_unstale":"2024-09-16T14:10:45.617104+0000","last_undegraded":"2024-09-16T14:10:45.617104+0000","last_fullsized":"2024-09-16T14:10:45.617104+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:08.523062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"197'667","reported_seq":15108,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589347+0000","last_change":"2024-09-16T14:10:29.448082+0000","last_active":"2024-09-16T14:10:45.589347+0000","last_peered":"2024-09-16T14:10:45.589347+0000","last_clean":"2024-09-16T14:10:45.589347+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:10:45.589347+0000","last_undegraded":"2024-09-16T14:10:45.589347+0000","last_fullsized":"2024-09-16T14:10:45.589347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:15.668078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020715640000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":14807,"num_read_kb":52699,"num_write":621,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617295+0000","last_change":"2024-09-16T14:10:00.112462+0000","last_active":"2024-09-16T14:10:45.617295+0000","last_peered":"2024-09-16T14:10:45.617295+0000","last_clean":"2024-09-16T14:10:45.617295+0000","last_became_active":"2024-09-16T14:10:00.105302+0000","last_became_peered":"2024-09-16T14:10:00.105302+0000","last_unstale":"2024-09-16T14:10:45.617295+0000","last_undegraded":"2024-09-16T14:10:45.617295+0000","last_fullsized":"2024-09-16T14:10:45.617295+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:34:48.094821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591766+0000","last_change":"2024-09-16T14:09:14.399362+0000","last_active":"2024-09-16T14:10:46.591766+0000","last_peered":"2024-09-16T14:10:46.591766+0000","last_clean":"2024-09-16T14:10:46.591766+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:10:46.591766+0000","last_undegraded":"2024-09-16T14:10:46.591766+0000","last_fullsized":"2024-09-16T14:10:46.591766+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:18.538987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"198'704","reported_seq":4286,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589250+0000","last_change":"2024-09-16T14:10:29.446791+0000","last_active":"2024-09-16T14:10:45.589250+0000","last_peered":"2024-09-16T14:10:45.589250+0000","last_clean":"2024-09-16T14:10:45.589250+0000","last_became_active":"2024-09-16T14:10:26.384950+0000","last_became_peered":"2024-09-16T14:10:26.384950+0000","last_unstale":"2024-09-16T14:10:45.589250+0000","last_undegraded":"2024-09-16T14:10:45.589250+0000","last_fullsized":"2024-09-16T14:10:45.589250+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:46.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088242400000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4245,"num_read_kb":11739,"num_write":722,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":2087145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"198'623","reported_seq":6962,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617420+0000","last_change":"2024-09-16T14:10:29.578958+0000","last_active":"2024-09-16T14:10:45.617420+0000","last_peered":"2024-09-16T14:10:45.617420+0000","last_clean":"2024-09-16T14:10:45.617420+0000","last_became_active":"2024-09-16T14:10:08.396086+0000","last_became_peered":"2024-09-16T14:10:08.396086+0000","last_unstale":"2024-09-16T14:10:45.617420+0000","last_undegraded":"2024-09-16T14:10:45.617420+0000","last_fullsized":"2024-09-16T14:10:45.617420+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:20.827364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00321149,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6657,"num_read_kb":16604,"num_write":570,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":10317335,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"197'850","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617685+0000","last_change":"2024-09-16T14:10:29.580121+0000","last_active":"2024-09-16T14:10:45.617685+0000","last_peered":"2024-09-16T14:10:45.617685+0000","last_clean":"2024-09-16T14:10:45.617685+0000","last_became_active":"2024-09-16T14:10:07.390964+0000","last_became_peered":"2024-09-16T14:10:07.390964+0000","last_unstale":"2024-09-16T14:10:45.617685+0000","last_undegraded":"2024-09-16T14:10:45.617685+0000","last_fullsized":"2024-09-16T14:10:45.617685+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:48.546737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041712620000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":7231,"num_read_kb":39110,"num_write":775,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":49748621,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"196'747","reported_seq":5182,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617930+0000","last_change":"2024-09-16T14:10:29.457917+0000","last_active":"2024-09-16T14:10:45.617930+0000","last_peered":"2024-09-16T14:10:45.617930+0000","last_clean":"2024-09-16T14:10:45.617930+0000","last_became_active":"2024-09-16T14:10:24.410718+0000","last_became_peered":"2024-09-16T14:10:24.410718+0000","last_unstale":"2024-09-16T14:10:45.617930+0000","last_undegraded":"2024-09-16T14:10:45.617930+0000","last_fullsized":"2024-09-16T14:10:45.617930+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:31.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064224900000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7094,"num_read_kb":44664,"num_write":801,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":19444502,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"197'621","reported_seq":8133,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.571439+0000","last_change":"2024-09-16T14:10:29.448787+0000","last_active":"2024-09-16T14:10:44.571439+0000","last_peered":"2024-09-16T14:10:44.571439+0000","last_clean":"2024-09-16T14:10:44.571439+0000","last_became_active":"2024-09-16T14:10:25.384634+0000","last_became_peered":"2024-09-16T14:10:25.384634+0000","last_unstale":"2024-09-16T14:10:44.571439+0000","last_undegraded":"2024-09-16T14:10:44.571439+0000","last_fullsized":"2024-09-16T14:10:44.571439+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:34.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034031370000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7377,"num_read_kb":27693,"num_write":494,"num_write_kb":13033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":23836787,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"197'496","reported_seq":2794,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617334+0000","last_change":"2024-09-16T14:10:29.581839+0000","last_active":"2024-09-16T14:10:45.617334+0000","last_peered":"2024-09-16T14:10:45.617334+0000","last_clean":"2024-09-16T14:10:45.617334+0000","last_became_active":"2024-09-16T14:10:04.123286+0000","last_became_peered":"2024-09-16T14:10:04.123286+0000","last_unstale":"2024-09-16T14:10:45.617334+0000","last_undegraded":"2024-09-16T14:10:45.617334+0000","last_fullsized":"2024-09-16T14:10:45.617334+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:39.608895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054803270000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":25169,"num_write":396,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":20371845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"198'510","reported_seq":7398,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573799+0000","last_change":"2024-09-16T14:10:29.446896+0000","last_active":"2024-09-16T14:10:44.573799+0000","last_peered":"2024-09-16T14:10:44.573799+0000","last_clean":"2024-09-16T14:10:44.573799+0000","last_became_active":"2024-09-16T14:10:16.487713+0000","last_became_peered":"2024-09-16T14:10:16.487713+0000","last_unstale":"2024-09-16T14:10:44.573799+0000","last_undegraded":"2024-09-16T14:10:44.573799+0000","last_fullsized":"2024-09-16T14:10:44.573799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:51.235912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072767700000000003,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7823,"num_read_kb":31641,"num_write":482,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"197'1003","reported_seq":6898,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.571831+0000","last_change":"2024-09-16T14:10:29.450285+0000","last_active":"2024-09-16T14:10:44.571831+0000","last_peered":"2024-09-16T14:10:44.571831+0000","last_clean":"2024-09-16T14:10:44.571831+0000","last_became_active":"2024-09-16T14:10:19.501036+0000","last_became_peered":"2024-09-16T14:10:19.501036+0000","last_unstale":"2024-09-16T14:10:44.571831+0000","last_undegraded":"2024-09-16T14:10:44.571831+0000","last_fullsized":"2024-09-16T14:10:44.571831+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:10.222500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041967539999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10106,"num_read_kb":24848,"num_write":1204,"num_write_kb":15655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":8219627,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"193'625","reported_seq":12640,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591811+0000","last_change":"2024-09-16T14:10:29.462983+0000","last_active":"2024-09-16T14:10:46.591811+0000","last_peered":"2024-09-16T14:10:46.591811+0000","last_clean":"2024-09-16T14:10:46.591811+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:10:46.591811+0000","last_undegraded":"2024-09-16T14:10:46.591811+0000","last_fullsized":"2024-09-16T14:10:46.591811+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:30.500273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114802,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13307,"num_read_kb":24046,"num_write":748,"num_write_kb":16341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"196'986","reported_seq":6243,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617242+0000","last_change":"2024-09-16T14:10:29.578998+0000","last_active":"2024-09-16T14:10:45.617242+0000","last_peered":"2024-09-16T14:10:45.617242+0000","last_clean":"2024-09-16T14:10:45.617242+0000","last_became_active":"2024-09-16T14:10:08.399369+0000","last_became_peered":"2024-09-16T14:10:08.399369+0000","last_unstale":"2024-09-16T14:10:45.617242+0000","last_undegraded":"2024-09-16T14:10:45.617242+0000","last_fullsized":"2024-09-16T14:10:45.617242+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:14.789387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036375169999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6839,"num_read_kb":16656,"num_write":1300,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":7816767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"198'741","reported_seq":6825,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616511+0000","last_change":"2024-09-16T14:10:29.576917+0000","last_active":"2024-09-16T14:10:45.616511+0000","last_peered":"2024-09-16T14:10:45.616511+0000","last_clean":"2024-09-16T14:10:45.616511+0000","last_became_active":"2024-09-16T14:10:05.121914+0000","last_became_peered":"2024-09-16T14:10:05.121914+0000","last_unstale":"2024-09-16T14:10:45.616511+0000","last_undegraded":"2024-09-16T14:10:45.616511+0000","last_fullsized":"2024-09-16T14:10:45.616511+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:38.113333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025226160000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7908,"num_read_kb":56867,"num_write":705,"num_write_kb":31085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":28924045,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"193'753","reported_seq":7845,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617139+0000","last_change":"2024-09-16T14:10:29.580755+0000","last_active":"2024-09-16T14:10:45.617139+0000","last_peered":"2024-09-16T14:10:45.617139+0000","last_clean":"2024-09-16T14:10:45.617139+0000","last_became_active":"2024-09-16T14:10:21.385616+0000","last_became_peered":"2024-09-16T14:10:21.385616+0000","last_unstale":"2024-09-16T14:10:45.617139+0000","last_undegraded":"2024-09-16T14:10:45.617139+0000","last_fullsized":"2024-09-16T14:10:45.617139+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:51.854260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047871169999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8482,"num_read_kb":31842,"num_write":873,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":41601355,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":588,"num_read":281369,"num_read_kb":918178,"num_write":31085,"num_write_kb":441735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9458,"num_bytes_recovered":336276108,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37982,"internal_metadata":0},"log_size":27091,"ondisk_log_size":27091,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304052,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067788,"statfs":{"total":96636764160,"available":96325414912,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.92100000000000004,"15min":0.90900000000000003},"min":{"1min":0.54700000000000004,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.4790000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.91100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.25,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.621}]},{"osd":1,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85199999999999998,"15min":0.85999999999999999},"min":{"1min":0.41199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0409999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.341,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1439999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.52900000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2529999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.371,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.76800000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0149999999999999,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2689999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.375},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.0089999999999999},"min":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.252}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.5549999999999999,"5min":3.246,"15min":3.246},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.024,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2909999999999999,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.52}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.0249999999999999},"min":{"1min":0.63700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.165,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.042},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0189999999999999},"min":{"1min":0.47199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5229999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":1.5840000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0840000000000001,"15min":1.069},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4650000000000001,"5min":3.492,"15min":3.492},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.123,"15min":1.131},"min":{"1min":0.59899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.577,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.6819999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312352,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059488,"statfs":{"total":96636764160,"available":96316915712,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1850000000000001,"5min":1.498,"15min":1.498},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2729999999999999,"5min":1.413,"15min":1.413},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95499999999999996,"15min":0.995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.77400000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84099999999999997,"15min":0.85699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.872},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.175,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.014}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.577,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88400000000000001,"15min":0.95399999999999996},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1200000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94199999999999995,"15min":0.97199999999999998},"min":{"1min":0.48299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.56100000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.976,"5min":4.976,"15min":4.976},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.004,"5min":1.03,"15min":1.0549999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.95599999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.55000000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.4319999999999999,"5min":3.4319999999999999,"15min":3.4319999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.101,"15min":1.1419999999999999},"min":{"1min":0.57399999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304116,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067724,"statfs":{"total":96636764160,"available":96325349376,"internally_reserved":0,"allocated":2654208,"data_stored":1841193,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97299999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.256},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.78700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.079}]},{"osd":1,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.85399999999999998,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.048,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90300000000000002,"15min":0.95499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0900000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.153}]},{"osd":2,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.94199999999999995,"15min":0.92300000000000004},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1479999999999999,"5min":10.999000000000001,"15min":10.999000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":11.018000000000001,"15min":11.018000000000001},"last":1.4159999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.212,"15min":1.212},"last":1.3089999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.93799999999999994},"min":{"1min":0.60399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.601,"15min":1.601},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.96699999999999997,"15min":0.996},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.244,"5min":1.764,"15min":1.764},"last":1.2070000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.538,"15min":1.538},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.999,"15min":1.0580000000000001},"min":{"1min":0.67200000000000004,"5min":0.315,"15min":0.315},"max":{"1min":1.409,"5min":1.573,"15min":1.573},"last":1.1339999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.008,"15min":1.046},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3100000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.0660000000000001,"15min":1.1220000000000001},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3759999999999999,"5min":1.619,"15min":1.619},"last":1.46}]}]},{"osd":3,"up_from":83,"seq":356482285598,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309916,"kb_used_data":2512,"kb_used_omap":31,"kb_used_meta":307360,"kb_avail":94061924,"statfs":{"total":96636764160,"available":96319410176,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31864,"internal_metadata":314737544},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.36699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":1.468,"15min":1.468},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1459999999999999,"5min":1.258,"15min":1.258},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93200000000000005,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.89600000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.119,"5min":1.347,"15min":1.347},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.96399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.2250000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95999999999999996,"15min":0.95299999999999996},"min":{"1min":0.64900000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.1439999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0349999999999999,"15min":1.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.03},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.042},"min":{"1min":0.72099999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.454,"5min":1.486,"15min":1.486},"last":1.202}]},{"osd":7,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.024,"15min":1.024},"min":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2929999999999999,"5min":1.419,"15min":1.419},"last":1.306},{"interface":"front","average":{"1min":1.103,"5min":1.0369999999999999,"15min":1.026},"min":{"1min":0.77500000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4139999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.2729999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308184,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063656,"statfs":{"total":96636764160,"available":96321183744,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.276,"5min":1.454,"15min":1.454},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Sep 16 14:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.5,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.753}]},{"osd":4,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.878}]},{"osd":5,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4390000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.139},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.948,"15min":1.948},"last":1.1839999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92700000000000005,"15min":0.93200000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.026,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.415,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.008}]},{"osd":7,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.98799999999999999,"15min":0.999},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.623,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.022,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4470000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.0469999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312424,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059416,"statfs":{"total":96636764160,"available":96316841984,"internally_reserved":0,"allocated":2637824,"data_stored":1841193,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.80800000000000005,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.5890000000000004,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.89800000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.70299999999999996,"15min":0.745},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.90500000000000003,"5min":1.998,"15min":1.998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Mon Sep 16 14:10:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":7.9950000000000001,"5min":7.9950000000000001,"15min":7.9950000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0009999999999999,"5min":1.244,"15min":1.244},"last":0.82199999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.89200000000000002},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.89600000000000002,"15min":0.92600000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1359999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.087,"15min":1.1200000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2330000000000001,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":1.034},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.077,"15min":1.097},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0600000000000001,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0429999999999999,"15min":1.0960000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.874}]},{"osd":7,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.018,"15min":1.0649999999999999},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.865,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.125},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.145,"15min":1.149},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.375,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":1.1930000000000001}]}]},{"osd":0,"up_from":184,"seq":790273982472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311836,"kb_used_data":2508,"kb_used_omap":125,"kb_used_meta":309186,"kb_avail":94060004,"statfs":{"total":96636764160,"available":96317444096,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":128171,"internal_metadata":316607317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4950000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6150000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316520,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94055320,"statfs":{"total":96636764160,"available":96312647680,"internally_reserved":0,"allocated":2580480,"data_stored":1245005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2809999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.268},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1990000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.2210000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.121,"5min":37.805,"15min":37.805},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1819999999999999,"15min":1.008},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.155,"5min":37.857999999999997,"15min":37.857999999999997},"last":0.91300000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.115,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.204,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1100000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:10:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.68000000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.133,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.60999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2829999999999999,"5min":1.611,"15min":1.611},"last":0.83499999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.95899999999999996,"15min":0.97299999999999998},"min":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3009999999999999,"5min":1.429,"15min":1.429},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.67700000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.278,"5min":1.601,"15min":1.601},"last":1.383}]},{"osd":7,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68700000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.355,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0780000000000001,"15min":1.097},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3819999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.236}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29034,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:10:50.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:10:51.113 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:51.114 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:10:51.126 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-16T14:10:50.830421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":588,"num_read":281435,"num_read_kb":918233,"num_write":31188,"num_write_kb":443365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9465,"num_bytes_recovered":336866533,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27166,"ondisk_log_size":27166,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2479400,"kb_used_data":20260,"kb_used_omap":166,"kb_used_meta":2458841,"kb_avail":752495320,"statfs":{"total":773094113280,"available":770555207680,"internally_reserved":0,"allocated":20746240,"data_stored":11135051,"data_compressed":11598,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":170259,"internal_metadata":2517853933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001577"},"pg_stats":[{"pgid":"3.1f","version":"198'1876","reported_seq":7887,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592881+0000","last_change":"2024-09-16T14:10:29.531167+0000","last_active":"2024-09-16T14:10:46.592881+0000","last_peered":"2024-09-16T14:10:46.592881+0000","last_clean":"2024-09-16T14:10:46.592881+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:10:46.592881+0000","last_undegraded":"2024-09-16T14:10:46.592881+0000","last_fullsized":"2024-09-16T14:10:46.592881+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1876,"log_dups_size":0,"ondisk_log_size":1876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:02.959438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039454019999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15599,"num_read_kb":19767,"num_write":2973,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"196'800","reported_seq":2726,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592990+0000","last_change":"2024-09-16T14:10:29.528090+0000","last_active":"2024-09-16T14:10:46.592990+0000","last_peered":"2024-09-16T14:10:46.592990+0000","last_clean":"2024-09-16T14:10:46.592990+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:10:46.592990+0000","last_undegraded":"2024-09-16T14:10:46.592990+0000","last_fullsized":"2024-09-16T14:10:46.592990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098036900000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3428,"num_read_kb":21398,"num_write":967,"num_write_kb":11613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"197'653","reported_seq":5530,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617422+0000","last_change":"2024-09-16T14:10:29.581527+0000","last_active":"2024-09-16T14:10:45.617422+0000","last_peered":"2024-09-16T14:10:45.617422+0000","last_clean":"2024-09-16T14:10:45.617422+0000","last_became_active":"2024-09-16T14:10:20.387629+0000","last_became_peered":"2024-09-16T14:10:20.387629+0000","last_unstale":"2024-09-16T14:10:45.617422+0000","last_undegraded":"2024-09-16T14:10:45.617422+0000","last_fullsized":"2024-09-16T14:10:45.617422+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:30.104851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051863780000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7537,"num_read_kb":24768,"num_write":780,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":26063912,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"198'1032","reported_seq":14527,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592725+0000","last_change":"2024-09-16T14:10:29.463373+0000","last_active":"2024-09-16T14:10:46.592725+0000","last_peered":"2024-09-16T14:10:46.592725+0000","last_clean":"2024-09-16T14:10:46.592725+0000","last_became_active":"2024-09-16T14:10:20.385216+0000","last_became_peered":"2024-09-16T14:10:20.385216+0000","last_unstale":"2024-09-16T14:10:46.592725+0000","last_undegraded":"2024-09-16T14:10:46.592725+0000","last_fullsized":"2024-09-16T14:10:46.592725+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:22.705027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013844,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":21,"num_read":15843,"num_read_kb":26936,"num_write":1369,"num_write_kb":11030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":6143753,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"197'738","reported_seq":8080,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.446977+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:20.387007+0000","last_became_peered":"2024-09-16T14:10:20.387007+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:41.660168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010460949999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10352,"num_read_kb":26550,"num_write":833,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"196'845","reported_seq":8027,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592011+0000","last_change":"2024-09-16T14:10:29.463712+0000","last_active":"2024-09-16T14:10:46.592011+0000","last_peered":"2024-09-16T14:10:46.592011+0000","last_clean":"2024-09-16T14:10:46.592011+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:10:46.592011+0000","last_undegraded":"2024-09-16T14:10:46.592011+0000","last_fullsized":"2024-09-16T14:10:46.592011+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:25.215853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017171580000000001,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":10449,"num_read_kb":28504,"num_write":803,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"197'630","reported_seq":2342,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589326+0000","last_change":"2024-09-16T14:10:29.447450+0000","last_active":"2024-09-16T14:10:45.589326+0000","last_peered":"2024-09-16T14:10:45.589326+0000","last_clean":"2024-09-16T14:10:45.589326+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:10:45.589326+0000","last_undegraded":"2024-09-16T14:10:45.589326+0000","last_fullsized":"2024-09-16T14:10:45.589326+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:41.369426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483112,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1924,"num_read_kb":28666,"num_write":709,"num_write_kb":21954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"198'968","reported_seq":6364,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592751+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592751+0000","last_peered":"2024-09-16T14:10:46.592751+0000","last_clean":"2024-09-16T14:10:46.592751+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:10:46.592751+0000","last_undegraded":"2024-09-16T14:10:46.592751+0000","last_fullsized":"2024-09-16T14:10:46.592751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050019210000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9238,"num_read_kb":21812,"num_write":1162,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"194'1351","reported_seq":6429,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593259+0000","last_change":"2024-09-16T14:10:29.531182+0000","last_active":"2024-09-16T14:10:46.593259+0000","last_peered":"2024-09-16T14:10:46.593259+0000","last_clean":"2024-09-16T14:10:46.593259+0000","last_became_active":"2024-09-16T14:10:05.123005+0000","last_became_peered":"2024-09-16T14:10:05.123005+0000","last_unstale":"2024-09-16T14:10:46.593259+0000","last_undegraded":"2024-09-16T14:10:46.593259+0000","last_fullsized":"2024-09-16T14:10:46.593259+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:44.294281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039366710000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8929,"num_read_kb":33272,"num_write":1987,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":4502377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"197'751","reported_seq":13941,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573574+0000","last_change":"2024-09-16T14:10:29.446978+0000","last_active":"2024-09-16T14:10:44.573574+0000","last_peered":"2024-09-16T14:10:44.573574+0000","last_clean":"2024-09-16T14:10:44.573574+0000","last_became_active":"2024-09-16T14:10:05.377635+0000","last_became_peered":"2024-09-16T14:10:05.377635+0000","last_unstale":"2024-09-16T14:10:44.573574+0000","last_undegraded":"2024-09-16T14:10:44.573574+0000","last_fullsized":"2024-09-16T14:10:44.573574+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:42.336862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089383699999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":14525,"num_read_kb":50269,"num_write":689,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":11596768,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"198'576","reported_seq":5189,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589895+0000","last_change":"2024-09-16T14:10:29.448044+0000","last_active":"2024-09-16T14:10:45.589895+0000","last_peered":"2024-09-16T14:10:45.589895+0000","last_clean":"2024-09-16T14:10:45.589895+0000","last_became_active":"2024-09-16T14:10:19.498121+0000","last_became_peered":"2024-09-16T14:10:19.498121+0000","last_unstale":"2024-09-16T14:10:45.589895+0000","last_undegraded":"2024-09-16T14:10:45.589895+0000","last_fullsized":"2024-09-16T14:10:45.589895+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:08.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002716544,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4971,"num_read_kb":28564,"num_write":500,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":50024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"197'723","reported_seq":7824,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617501+0000","last_change":"2024-09-16T14:10:29.582367+0000","last_active":"2024-09-16T14:10:45.617501+0000","last_peered":"2024-09-16T14:10:45.617501+0000","last_clean":"2024-09-16T14:10:45.617501+0000","last_became_active":"2024-09-16T14:10:06.388650+0000","last_became_peered":"2024-09-16T14:10:06.388650+0000","last_unstale":"2024-09-16T14:10:45.617501+0000","last_undegraded":"2024-09-16T14:10:45.617501+0000","last_fullsized":"2024-09-16T14:10:45.617501+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:30.578665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054744670000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9236,"num_read_kb":38811,"num_write":682,"num_write_kb":10309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":10174643,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617632+0000","last_change":"2024-09-16T14:10:17.556015+0000","last_active":"2024-09-16T14:10:45.617632+0000","last_peered":"2024-09-16T14:10:45.617632+0000","last_clean":"2024-09-16T14:10:45.617632+0000","last_became_active":"2024-09-16T14:10:17.555698+0000","last_became_peered":"2024-09-16T14:10:17.555698+0000","last_unstale":"2024-09-16T14:10:45.617632+0000","last_undegraded":"2024-09-16T14:10:45.617632+0000","last_fullsized":"2024-09-16T14:10:45.617632+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:03.785460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"198'897","reported_seq":6176,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.448811+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:25.385245+0000","last_became_peered":"2024-09-16T14:10:25.385245+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:00.028708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026361219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5530,"num_read_kb":25811,"num_write":916,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":731,"num_bytes_recovered":15701912,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589505+0000","last_change":"2024-09-16T14:10:16.472462+0000","last_active":"2024-09-16T14:10:45.589505+0000","last_peered":"2024-09-16T14:10:45.589505+0000","last_clean":"2024-09-16T14:10:45.589505+0000","last_became_active":"2024-09-16T14:10:16.471977+0000","last_became_peered":"2024-09-16T14:10:16.471977+0000","last_unstale":"2024-09-16T14:10:45.589505+0000","last_undegraded":"2024-09-16T14:10:45.589505+0000","last_fullsized":"2024-09-16T14:10:45.589505+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:09.028511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"196'967","reported_seq":7724,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589578+0000","last_change":"2024-09-16T14:10:29.448302+0000","last_active":"2024-09-16T14:10:45.589578+0000","last_peered":"2024-09-16T14:10:45.589578+0000","last_clean":"2024-09-16T14:10:45.589578+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:10:45.589578+0000","last_undegraded":"2024-09-16T14:10:45.589578+0000","last_fullsized":"2024-09-16T14:10:45.589578+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:58.173875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002259535,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":10392,"num_read_kb":45651,"num_write":1244,"num_write_kb":30465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617428+0000","last_change":"2024-09-16T14:10:00.102785+0000","last_active":"2024-09-16T14:10:45.617428+0000","last_peered":"2024-09-16T14:10:45.617428+0000","last_clean":"2024-09-16T14:10:45.617428+0000","last_became_active":"2024-09-16T14:10:00.102521+0000","last_became_peered":"2024-09-16T14:10:00.102521+0000","last_unstale":"2024-09-16T14:10:45.617428+0000","last_undegraded":"2024-09-16T14:10:45.617428+0000","last_fullsized":"2024-09-16T14:10:45.617428+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:17.608389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"197'630","reported_seq":3952,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616832+0000","last_change":"2024-09-16T14:10:29.574272+0000","last_active":"2024-09-16T14:10:45.616832+0000","last_peered":"2024-09-16T14:10:45.616832+0000","last_clean":"2024-09-16T14:10:45.616832+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:10:45.616832+0000","last_undegraded":"2024-09-16T14:10:45.616832+0000","last_fullsized":"2024-09-16T14:10:45.616832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001012095,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4560,"num_read_kb":16131,"num_write":597,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593236+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:10:46.593236+0000","last_peered":"2024-09-16T14:10:46.593236+0000","last_clean":"2024-09-16T14:10:46.593236+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:10:46.593236+0000","last_undegraded":"2024-09-16T14:10:46.593236+0000","last_fullsized":"2024-09-16T14:10:46.593236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"197'780","reported_seq":9812,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573395+0000","last_change":"2024-09-16T14:10:29.447893+0000","last_active":"2024-09-16T14:10:44.573395+0000","last_peered":"2024-09-16T14:10:44.573395+0000","last_clean":"2024-09-16T14:10:44.573395+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:10:44.573395+0000","last_undegraded":"2024-09-16T14:10:44.573395+0000","last_fullsized":"2024-09-16T14:10:44.573395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064281,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":11041,"num_read_kb":12619,"num_write":857,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"198'1412","reported_seq":5233,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590076+0000","last_change":"2024-09-16T14:10:29.449673+0000","last_active":"2024-09-16T14:10:45.590076+0000","last_peered":"2024-09-16T14:10:45.590076+0000","last_clean":"2024-09-16T14:10:45.590076+0000","last_became_active":"2024-09-16T14:10:19.505692+0000","last_became_peered":"2024-09-16T14:10:19.505692+0000","last_unstale":"2024-09-16T14:10:45.590076+0000","last_undegraded":"2024-09-16T14:10:45.590076+0000","last_fullsized":"2024-09-16T14:10:45.590076+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:54.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040141910000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":9687,"num_read_kb":19616,"num_write":1641,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617596+0000","last_change":"2024-09-16T14:10:00.110539+0000","last_active":"2024-09-16T14:10:45.617596+0000","last_peered":"2024-09-16T14:10:45.617596+0000","last_clean":"2024-09-16T14:10:45.617596+0000","last_became_active":"2024-09-16T14:10:00.104539+0000","last_became_peered":"2024-09-16T14:10:00.104539+0000","last_unstale":"2024-09-16T14:10:45.617596+0000","last_undegraded":"2024-09-16T14:10:45.617596+0000","last_fullsized":"2024-09-16T14:10:45.617596+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:07.278197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"196'1316","reported_seq":6546,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592682+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592682+0000","last_peered":"2024-09-16T14:10:46.592682+0000","last_clean":"2024-09-16T14:10:46.592682+0000","last_became_active":"2024-09-16T14:10:26.385527+0000","last_became_peered":"2024-09-16T14:10:26.385527+0000","last_unstale":"2024-09-16T14:10:46.592682+0000","last_undegraded":"2024-09-16T14:10:46.592682+0000","last_fullsized":"2024-09-16T14:10:46.592682+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1316,"log_dups_size":0,"ondisk_log_size":1316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:03.910845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051618139999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9488,"num_read_kb":24272,"num_write":1738,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":8428888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617802+0000","last_change":"2024-09-16T14:10:21.388025+0000","last_active":"2024-09-16T14:10:45.617802+0000","last_peered":"2024-09-16T14:10:45.617802+0000","last_clean":"2024-09-16T14:10:45.617802+0000","last_became_active":"2024-09-16T14:10:21.387173+0000","last_became_peered":"2024-09-16T14:10:21.387173+0000","last_unstale":"2024-09-16T14:10:45.617802+0000","last_undegraded":"2024-09-16T14:10:45.617802+0000","last_fullsized":"2024-09-16T14:10:45.617802+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:57.862024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"197'820","reported_seq":7030,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616505+0000","last_change":"2024-09-16T14:10:29.574667+0000","last_active":"2024-09-16T14:10:45.616505+0000","last_peered":"2024-09-16T14:10:45.616505+0000","last_clean":"2024-09-16T14:10:45.616505+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:10:45.616505+0000","last_undegraded":"2024-09-16T14:10:45.616505+0000","last_fullsized":"2024-09-16T14:10:45.616505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233734,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":8272,"num_read_kb":21183,"num_write":947,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617104+0000","last_change":"2024-09-16T14:10:00.104224+0000","last_active":"2024-09-16T14:10:45.617104+0000","last_peered":"2024-09-16T14:10:45.617104+0000","last_clean":"2024-09-16T14:10:45.617104+0000","last_became_active":"2024-09-16T14:10:00.104003+0000","last_became_peered":"2024-09-16T14:10:00.104003+0000","last_unstale":"2024-09-16T14:10:45.617104+0000","last_undegraded":"2024-09-16T14:10:45.617104+0000","last_fullsized":"2024-09-16T14:10:45.617104+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:08.523062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"197'667","reported_seq":15108,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589347+0000","last_change":"2024-09-16T14:10:29.448082+0000","last_active":"2024-09-16T14:10:45.589347+0000","last_peered":"2024-09-16T14:10:45.589347+0000","last_clean":"2024-09-16T14:10:45.589347+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:10:45.589347+0000","last_undegraded":"2024-09-16T14:10:45.589347+0000","last_fullsized":"2024-09-16T14:10:45.589347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:15.668078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020715640000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":14807,"num_read_kb":52699,"num_write":621,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617295+0000","last_change":"2024-09-16T14:10:00.112462+0000","last_active":"2024-09-16T14:10:45.617295+0000","last_peered":"2024-09-16T14:10:45.617295+0000","last_clean":"2024-09-16T14:10:45.617295+0000","last_became_active":"2024-09-16T14:10:00.105302+0000","last_became_peered":"2024-09-16T14:10:00.105302+0000","last_unstale":"2024-09-16T14:10:45.617295+0000","last_undegraded":"2024-09-16T14:10:45.617295+0000","last_fullsized":"2024-09-16T14:10:45.617295+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:34:48.094821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591766+0000","last_change":"2024-09-16T14:09:14.399362+0000","last_active":"2024-09-16T14:10:46.591766+0000","last_peered":"2024-09-16T14:10:46.591766+0000","last_clean":"2024-09-16T14:10:46.591766+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:10:46.591766+0000","last_undegraded":"2024-09-16T14:10:46.591766+0000","last_fullsized":"2024-09-16T14:10:46.591766+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:18.538987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"198'704","reported_seq":4286,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589250+0000","last_change":"2024-09-16T14:10:29.446791+0000","last_active":"2024-09-16T14:10:45.589250+0000","last_peered":"2024-09-16T14:10:45.589250+0000","last_clean":"2024-09-16T14:10:45.589250+0000","last_became_active":"2024-09-16T14:10:26.384950+0000","last_became_peered":"2024-09-16T14:10:26.384950+0000","last_unstale":"2024-09-16T14:10:45.589250+0000","last_undegraded":"2024-09-16T14:10:45.589250+0000","last_fullsized":"2024-09-16T14:10:45.589250+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:46.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088242400000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4245,"num_read_kb":11739,"num_write":722,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":2087145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"198'623","reported_seq":6962,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617420+0000","last_change":"2024-09-16T14:10:29.578958+0000","last_active":"2024-09-16T14:10:45.617420+0000","last_peered":"2024-09-16T14:10:45.617420+0000","last_clean":"2024-09-16T14:10:45.617420+0000","last_became_active":"2024-09-16T14:10:08.396086+0000","last_became_peered":"2024-09-16T14:10:08.396086+0000","last_unstale":"2024-09-16T14:10:45.617420+0000","last_undegraded":"2024-09-16T14:10:45.617420+0000","last_fullsized":"2024-09-16T14:10:45.617420+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:20.827364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00321149,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6657,"num_read_kb":16604,"num_write":570,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":10317335,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"197'850","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617685+0000","last_change":"2024-09-16T14:10:29.580121+0000","last_active":"2024-09-16T14:10:45.617685+0000","last_peered":"2024-09-16T14:10:45.617685+0000","last_clean":"2024-09-16T14:10:45.617685+0000","last_became_active":"2024-09-16T14:10:07.390964+0000","last_became_peered":"2024-09-16T14:10:07.390964+0000","last_unstale":"2024-09-16T14:10:45.617685+0000","last_undegraded":"2024-09-16T14:10:45.617685+0000","last_fullsized":"2024-09-16T14:10:45.617685+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:48.546737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041712620000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":7231,"num_read_kb":39110,"num_write":775,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":49748621,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"196'747","reported_seq":5182,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617930+0000","last_change":"2024-09-16T14:10:29.457917+0000","last_active":"2024-09-16T14:10:45.617930+0000","last_peered":"2024-09-16T14:10:45.617930+0000","last_clean":"2024-09-16T14:10:45.617930+0000","last_became_active":"2024-09-16T14:10:24.410718+0000","last_became_peered":"2024-09-16T14:10:24.410718+0000","last_unstale":"2024-09-16T14:10:45.617930+0000","last_undegraded":"2024-09-16T14:10:45.617930+0000","last_fullsized":"2024-09-16T14:10:45.617930+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:31.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064224900000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7094,"num_read_kb":44664,"num_write":801,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":19444502,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"197'621","reported_seq":8134,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590037+0000","last_change":"2024-09-16T14:10:29.448787+0000","last_active":"2024-09-16T14:10:45.590037+0000","last_peered":"2024-09-16T14:10:45.590037+0000","last_clean":"2024-09-16T14:10:45.590037+0000","last_became_active":"2024-09-16T14:10:25.384634+0000","last_became_peered":"2024-09-16T14:10:25.384634+0000","last_unstale":"2024-09-16T14:10:45.590037+0000","last_undegraded":"2024-09-16T14:10:45.590037+0000","last_fullsized":"2024-09-16T14:10:45.590037+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:34.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034031370000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7377,"num_read_kb":27693,"num_write":494,"num_write_kb":13033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":23836787,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"197'496","reported_seq":2794,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617334+0000","last_change":"2024-09-16T14:10:29.581839+0000","last_active":"2024-09-16T14:10:45.617334+0000","last_peered":"2024-09-16T14:10:45.617334+0000","last_clean":"2024-09-16T14:10:45.617334+0000","last_became_active":"2024-09-16T14:10:04.123286+0000","last_became_peered":"2024-09-16T14:10:04.123286+0000","last_unstale":"2024-09-16T14:10:45.617334+0000","last_undegraded":"2024-09-16T14:10:45.617334+0000","last_fullsized":"2024-09-16T14:10:45.617334+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:39.608895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054803270000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":25169,"num_write":396,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":20371845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"198'510","reported_seq":7398,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573799+0000","last_change":"2024-09-16T14:10:29.446896+0000","last_active":"2024-09-16T14:10:44.573799+0000","last_peered":"2024-09-16T14:10:44.573799+0000","last_clean":"2024-09-16T14:10:44.573799+0000","last_became_active":"2024-09-16T14:10:16.487713+0000","last_became_peered":"2024-09-16T14:10:16.487713+0000","last_unstale":"2024-09-16T14:10:44.573799+0000","last_undegraded":"2024-09-16T14:10:44.573799+0000","last_fullsized":"2024-09-16T14:10:44.573799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:51.235912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072767700000000003,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7823,"num_read_kb":31641,"num_write":482,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"197'1003","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590342+0000","last_change":"2024-09-16T14:10:29.450285+0000","last_active":"2024-09-16T14:10:45.590342+0000","last_peered":"2024-09-16T14:10:45.590342+0000","last_clean":"2024-09-16T14:10:45.590342+0000","last_became_active":"2024-09-16T14:10:19.501036+0000","last_became_peered":"2024-09-16T14:10:19.501036+0000","last_unstale":"2024-09-16T14:10:45.590342+0000","last_undegraded":"2024-09-16T14:10:45.590342+0000","last_fullsized":"2024-09-16T14:10:45.590342+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:10.222500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041967539999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10106,"num_read_kb":24848,"num_write":1204,"num_write_kb":15655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":8219627,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"193'625","reported_seq":12640,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591811+0000","last_change":"2024-09-16T14:10:29.462983+0000","last_active":"2024-09-16T14:10:46.591811+0000","last_peered":"2024-09-16T14:10:46.591811+0000","last_clean":"2024-09-16T14:10:46.591811+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:10:46.591811+0000","last_undegraded":"2024-09-16T14:10:46.591811+0000","last_fullsized":"2024-09-16T14:10:46.591811+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:30.500273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114802,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13307,"num_read_kb":24046,"num_write":748,"num_write_kb":16341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"196'986","reported_seq":6243,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617242+0000","last_change":"2024-09-16T14:10:29.578998+0000","last_active":"2024-09-16T14:10:45.617242+0000","last_peered":"2024-09-16T14:10:45.617242+0000","last_clean":"2024-09-16T14:10:45.617242+0000","last_became_active":"2024-09-16T14:10:08.399369+0000","last_became_peered":"2024-09-16T14:10:08.399369+0000","last_unstale":"2024-09-16T14:10:45.617242+0000","last_undegraded":"2024-09-16T14:10:45.617242+0000","last_fullsized":"2024-09-16T14:10:45.617242+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:14.789387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036375169999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6839,"num_read_kb":16656,"num_write":1300,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":7816767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"198'741","reported_seq":6825,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616511+0000","last_change":"2024-09-16T14:10:29.576917+0000","last_active":"2024-09-16T14:10:45.616511+0000","last_peered":"2024-09-16T14:10:45.616511+0000","last_clean":"2024-09-16T14:10:45.616511+0000","last_became_active":"2024-09-16T14:10:05.121914+0000","last_became_peered":"2024-09-16T14:10:05.121914+0000","last_unstale":"2024-09-16T14:10:45.616511+0000","last_undegraded":"2024-09-16T14:10:45.616511+0000","last_fullsized":"2024-09-16T14:10:45.616511+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:38.113333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025226160000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7908,"num_read_kb":56867,"num_write":705,"num_write_kb":31085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":28924045,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"193'753","reported_seq":7845,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617139+0000","last_change":"2024-09-16T14:10:29.580755+0000","last_active":"2024-09-16T14:10:45.617139+0000","last_peered":"2024-09-16T14:10:45.617139+0000","last_clean":"2024-09-16T14:10:45.617139+0000","last_became_active":"2024-09-16T14:10:21.385616+0000","last_became_peered":"2024-09-16T14:10:21.385616+0000","last_unstale":"2024-09-16T14:10:45.617139+0000","last_undegraded":"2024-09-16T14:10:45.617139+0000","last_fullsized":"2024-09-16T14:10:45.617139+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:51.854260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047871169999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8482,"num_read_kb":31842,"num_write":873,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":41601355,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":588,"num_read":281369,"num_read_kb":918178,"num_write":31085,"num_write_kb":441735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9458,"num_bytes_recovered":336276108,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37982,"internal_metadata":0},"log_size":27091,"ondisk_log_size":27091,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304052,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067788,"statfs":{"total":96636764160,"available":96325414912,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.92100000000000004,"15min":0.90900000000000003},"min":{"1min":0.54700000000000004,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.4790000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.91100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.25,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.621}]},{"osd":1,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85199999999999998,"15min":0.85999999999999999},"min":{"1min":0.41199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0409999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.341,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1439999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.52900000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2529999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.371,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.76800000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0149999999999999,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2689999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.375},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.0089999999999999},"min":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.252}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.5549999999999999,"5min":3.246,"15min":3.246},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.024,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2909999999999999,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.52}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.0249999999999999},"min":{"1min":0.63700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.165,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.042},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0189999999999999},"min":{"1min":0.47199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5229999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":1.5840000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0840000000000001,"15min":1.069},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4650000000000001,"5min":3.492,"15min":3.492},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.123,"15min":1.131},"min":{"1min":0.59899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.577,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.6819999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312352,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059488,"statfs":{"total":96636764160,"available":96316915712,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1850000000000001,"5min":1.498,"15min":1.498},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2729999999999999,"5min":1.413,"15min":1.413},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95499999999999996,"15min":0.995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.77400000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84099999999999997,"15min":0.85699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.872},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.175,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.014}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.577,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88400000000000001,"15min":0.95399999999999996},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1200000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94199999999999995,"15min":0.97199999999999998},"min":{"1min":0.48299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.56100000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.976,"5min":4.976,"15min":4.976},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.004,"5min":1.03,"15min":1.0549999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.95599999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.55000000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.4319999999999999,"5min":3.4319999999999999,"15min":3.4319999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.101,"15min":1.1419999999999999},"min":{"1min":0.57399999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304116,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067724,"statfs":{"total":96636764160,"available":96325349376,"internally_reserved":0,"allocated":2654208,"data_stored":1841193,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97299999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.256},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.78700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.079}]},{"osd":1,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.85399999999999998,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.048,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90300000000000002,"15min":0.95499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0900000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.153}]},{"osd":2,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.94199999999999995,"15min":0.92300000000000004},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1479999999999999,"5min":10.999000000000001,"15min":10.999000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":11.018000000000001,"15min":11.018000000000001},"last":1.4159999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.212,"15min":1.212},"last":1.3089999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.93799999999999994},"min":{"1min":0.60399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.601,"15min":1.601},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.96699999999999997,"15min":0.996},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.244,"5min":1.764,"15min":1.764},"last":1.2070000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.538,"15min":1.538},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.999,"15min":1.0580000000000001},"min":{"1min":0.67200000000000004,"5min":0.315,"15min":0.315},"max":{"1min":1.409,"5min":1.573,"15min":1.573},"last":1.1339999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.008,"15min":1.046},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3100000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.0660000000000001,"15min":1.1220000000000001},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3759999999999999,"5min":1.619,"15min":1.619},"last":1.46}]}]},{"osd":3,"up_from":83,"seq":356482285598,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309916,"kb_used_data":2512,"kb_used_omap":31,"kb_used_meta":307360,"kb_avail":94061924,"statfs":{"total":96636764160,"available":96319410176,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31864,"internal_metadata":314737544},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.36699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":1.468,"15min":1.468},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1459999999999999,"5min":1.258,"15min":1.258},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93200000000000005,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.89600000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.119,"5min":1.347,"15min":1.347},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.96399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.2250000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95999999999999996,"15min":0.95299999999999996},"min":{"1min":0.64900000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.1439999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0349999999999999,"15min":1.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.03},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.042},"min":{"1min":0.72099999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.454,"5min":1.486,"15min":1.486},"last":1.202}]},{"osd":7,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.024,"15min":1.024},"min":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2929999999999999,"5min":1.419,"15min":1.419},"last":1.306},{"interface":"front","average":{"1min":1.103,"5min":1.0369999999999999,"15min":1.026},"min":{"1min":0.77500000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4139999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.2729999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308184,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063656,"statfs":{"total":96636764160,"available":96321183744,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.276,"5min":1.454,"15min":1.454},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Sep 16 14:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.5,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.753}]},{"osd":4,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.878}]},{"osd":5,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4390000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.139},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.948,"15min":1.948},"last":1.1839999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92700000000000005,"15min":0.93200000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.026,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.415,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.008}]},{"osd":7,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.98799999999999999,"15min":0.999},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.623,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.022,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4470000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.0469999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312424,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059416,"statfs":{"total":96636764160,"available":96316841984,"internally_reserved":0,"allocated":2637824,"data_stored":1841193,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.80800000000000005,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.5890000000000004,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.89800000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.70299999999999996,"15min":0.745},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.90500000000000003,"5min":1.998,"15min":1.998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Mon Sep 16 14:10:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":7.9950000000000001,"5min":7.9950000000000001,"15min":7.9950000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0009999999999999,"5min":1.244,"15min":1.244},"last":0.82199999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.89200000000000002},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.89600000000000002,"15min":0.92600000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1359999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.087,"15min":1.1200000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2330000000000001,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":1.034},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.077,"15min":1.097},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0600000000000001,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0429999999999999,"15min":1.0960000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.874}]},{"osd":7,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.018,"15min":1.0649999999999999},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.865,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.125},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.145,"15min":1.149},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.375,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":1.1930000000000001}]}]},{"osd":0,"up_from":184,"seq":790273982473,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311836,"kb_used_data":2508,"kb_used_omap":125,"kb_used_meta":309186,"kb_avail":94060004,"statfs":{"total":96636764160,"available":96317444096,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":128171,"internal_metadata":316607317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316520,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94055320,"statfs":{"total":96636764160,"available":96312647680,"internally_reserved":0,"allocated":2580480,"data_stored":1245005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2809999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.268},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1990000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.2210000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.121,"5min":37.805,"15min":37.805},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1819999999999999,"15min":1.008},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.155,"5min":37.857999999999997,"15min":37.857999999999997},"last":0.91300000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.115,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.204,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1100000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:10:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.68000000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.133,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.60999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2829999999999999,"5min":1.611,"15min":1.611},"last":0.83499999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.95899999999999996,"15min":0.97299999999999998},"min":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3009999999999999,"5min":1.429,"15min":1.429},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.67700000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.278,"5min":1.601,"15min":1.601},"last":1.383}]},{"osd":7,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68700000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.355,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0780000000000001,"15min":1.097},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3819999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.236}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29034,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:10:51.129 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T14:10:51.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T14:10:51.424 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T14:10:51.433 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-16T14:10:51.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:10:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:51.724 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":215,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:10:47.595917+0000","last_up_change":"2024-09-16T14:10:15.439434+0000","last_in_change":"2024-09-16T14:10:20.368208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:06:58.526935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T14:07:08.375919+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":103,"snap_epoch":198,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":129,"up_from":184,"up_thru":193,"down_at":130,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":47342},{"type":"v1","addr":"172.21.15.79:6817","nonce":47342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":47342},{"type":"v1","addr":"172.21.15.79:6819","nonce":47342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":47342},{"type":"v1","addr":"172.21.15.79:6824","nonce":47342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":47342},{"type":"v1","addr":"172.21.15.79:6821","nonce":47342}]},"public_addr":"172.21.15.79:6817/47342","cluster_addr":"172.21.15.79:6819/47342","heartbeat_back_addr":"172.21.15.79:6824/47342","heartbeat_front_addr":"172.21.15.79:6821/47342","state":["exists","up"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":83,"up_thru":189,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":35927},{"type":"v1","addr":"172.21.15.79:6801","nonce":35927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":35927},{"type":"v1","addr":"172.21.15.79:6803","nonce":35927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":35927},{"type":"v1","addr":"172.21.15.79:6807","nonce":35927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":35927},{"type":"v1","addr":"172.21.15.79:6805","nonce":35927}]},"public_addr":"172.21.15.79:6801/35927","cluster_addr":"172.21.15.79:6803/35927","heartbeat_back_addr":"172.21.15.79:6807/35927","heartbeat_front_addr":"172.21.15.79:6805/35927","state":["exists","up"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":130},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":59},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:10:51.738 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T14:10:51.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:10:52.016 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:52.019 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:10:52.031 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-16T14:10:50.830421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":588,"num_read":281435,"num_read_kb":918233,"num_write":31188,"num_write_kb":443365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9465,"num_bytes_recovered":336866533,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27166,"ondisk_log_size":27166,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2479400,"kb_used_data":20260,"kb_used_omap":166,"kb_used_meta":2458841,"kb_avail":752495320,"statfs":{"total":773094113280,"available":770555207680,"internally_reserved":0,"allocated":20746240,"data_stored":11135051,"data_compressed":11598,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":170259,"internal_metadata":2517853933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001577"},"pg_stats":[{"pgid":"3.1f","version":"198'1876","reported_seq":7887,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592881+0000","last_change":"2024-09-16T14:10:29.531167+0000","last_active":"2024-09-16T14:10:46.592881+0000","last_peered":"2024-09-16T14:10:46.592881+0000","last_clean":"2024-09-16T14:10:46.592881+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:10:46.592881+0000","last_undegraded":"2024-09-16T14:10:46.592881+0000","last_fullsized":"2024-09-16T14:10:46.592881+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1876,"log_dups_size":0,"ondisk_log_size":1876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:02.959438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039454019999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15599,"num_read_kb":19767,"num_write":2973,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"196'800","reported_seq":2726,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592990+0000","last_change":"2024-09-16T14:10:29.528090+0000","last_active":"2024-09-16T14:10:46.592990+0000","last_peered":"2024-09-16T14:10:46.592990+0000","last_clean":"2024-09-16T14:10:46.592990+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:10:46.592990+0000","last_undegraded":"2024-09-16T14:10:46.592990+0000","last_fullsized":"2024-09-16T14:10:46.592990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098036900000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3428,"num_read_kb":21398,"num_write":967,"num_write_kb":11613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"197'653","reported_seq":5530,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617422+0000","last_change":"2024-09-16T14:10:29.581527+0000","last_active":"2024-09-16T14:10:45.617422+0000","last_peered":"2024-09-16T14:10:45.617422+0000","last_clean":"2024-09-16T14:10:45.617422+0000","last_became_active":"2024-09-16T14:10:20.387629+0000","last_became_peered":"2024-09-16T14:10:20.387629+0000","last_unstale":"2024-09-16T14:10:45.617422+0000","last_undegraded":"2024-09-16T14:10:45.617422+0000","last_fullsized":"2024-09-16T14:10:45.617422+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:30.104851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051863780000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7537,"num_read_kb":24768,"num_write":780,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":26063912,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"198'1032","reported_seq":14527,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592725+0000","last_change":"2024-09-16T14:10:29.463373+0000","last_active":"2024-09-16T14:10:46.592725+0000","last_peered":"2024-09-16T14:10:46.592725+0000","last_clean":"2024-09-16T14:10:46.592725+0000","last_became_active":"2024-09-16T14:10:20.385216+0000","last_became_peered":"2024-09-16T14:10:20.385216+0000","last_unstale":"2024-09-16T14:10:46.592725+0000","last_undegraded":"2024-09-16T14:10:46.592725+0000","last_fullsized":"2024-09-16T14:10:46.592725+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:22.705027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013844,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":21,"num_read":15843,"num_read_kb":26936,"num_write":1369,"num_write_kb":11030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":6143753,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"197'738","reported_seq":8080,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.446977+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:20.387007+0000","last_became_peered":"2024-09-16T14:10:20.387007+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:41.660168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010460949999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10352,"num_read_kb":26550,"num_write":833,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"196'845","reported_seq":8027,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592011+0000","last_change":"2024-09-16T14:10:29.463712+0000","last_active":"2024-09-16T14:10:46.592011+0000","last_peered":"2024-09-16T14:10:46.592011+0000","last_clean":"2024-09-16T14:10:46.592011+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:10:46.592011+0000","last_undegraded":"2024-09-16T14:10:46.592011+0000","last_fullsized":"2024-09-16T14:10:46.592011+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:25.215853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017171580000000001,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":10449,"num_read_kb":28504,"num_write":803,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"197'630","reported_seq":2342,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589326+0000","last_change":"2024-09-16T14:10:29.447450+0000","last_active":"2024-09-16T14:10:45.589326+0000","last_peered":"2024-09-16T14:10:45.589326+0000","last_clean":"2024-09-16T14:10:45.589326+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:10:45.589326+0000","last_undegraded":"2024-09-16T14:10:45.589326+0000","last_fullsized":"2024-09-16T14:10:45.589326+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:41.369426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483112,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1924,"num_read_kb":28666,"num_write":709,"num_write_kb":21954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"198'968","reported_seq":6364,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592751+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592751+0000","last_peered":"2024-09-16T14:10:46.592751+0000","last_clean":"2024-09-16T14:10:46.592751+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:10:46.592751+0000","last_undegraded":"2024-09-16T14:10:46.592751+0000","last_fullsized":"2024-09-16T14:10:46.592751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050019210000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9238,"num_read_kb":21812,"num_write":1162,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"194'1351","reported_seq":6429,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593259+0000","last_change":"2024-09-16T14:10:29.531182+0000","last_active":"2024-09-16T14:10:46.593259+0000","last_peered":"2024-09-16T14:10:46.593259+0000","last_clean":"2024-09-16T14:10:46.593259+0000","last_became_active":"2024-09-16T14:10:05.123005+0000","last_became_peered":"2024-09-16T14:10:05.123005+0000","last_unstale":"2024-09-16T14:10:46.593259+0000","last_undegraded":"2024-09-16T14:10:46.593259+0000","last_fullsized":"2024-09-16T14:10:46.593259+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:44.294281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039366710000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8929,"num_read_kb":33272,"num_write":1987,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":4502377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"197'751","reported_seq":13941,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573574+0000","last_change":"2024-09-16T14:10:29.446978+0000","last_active":"2024-09-16T14:10:44.573574+0000","last_peered":"2024-09-16T14:10:44.573574+0000","last_clean":"2024-09-16T14:10:44.573574+0000","last_became_active":"2024-09-16T14:10:05.377635+0000","last_became_peered":"2024-09-16T14:10:05.377635+0000","last_unstale":"2024-09-16T14:10:44.573574+0000","last_undegraded":"2024-09-16T14:10:44.573574+0000","last_fullsized":"2024-09-16T14:10:44.573574+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:42.336862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089383699999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":14525,"num_read_kb":50269,"num_write":689,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":11596768,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"198'576","reported_seq":5189,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589895+0000","last_change":"2024-09-16T14:10:29.448044+0000","last_active":"2024-09-16T14:10:45.589895+0000","last_peered":"2024-09-16T14:10:45.589895+0000","last_clean":"2024-09-16T14:10:45.589895+0000","last_became_active":"2024-09-16T14:10:19.498121+0000","last_became_peered":"2024-09-16T14:10:19.498121+0000","last_unstale":"2024-09-16T14:10:45.589895+0000","last_undegraded":"2024-09-16T14:10:45.589895+0000","last_fullsized":"2024-09-16T14:10:45.589895+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:08.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002716544,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4971,"num_read_kb":28564,"num_write":500,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":50024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"197'723","reported_seq":7824,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617501+0000","last_change":"2024-09-16T14:10:29.582367+0000","last_active":"2024-09-16T14:10:45.617501+0000","last_peered":"2024-09-16T14:10:45.617501+0000","last_clean":"2024-09-16T14:10:45.617501+0000","last_became_active":"2024-09-16T14:10:06.388650+0000","last_became_peered":"2024-09-16T14:10:06.388650+0000","last_unstale":"2024-09-16T14:10:45.617501+0000","last_undegraded":"2024-09-16T14:10:45.617501+0000","last_fullsized":"2024-09-16T14:10:45.617501+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:30.578665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054744670000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9236,"num_read_kb":38811,"num_write":682,"num_write_kb":10309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":10174643,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617632+0000","last_change":"2024-09-16T14:10:17.556015+0000","last_active":"2024-09-16T14:10:45.617632+0000","last_peered":"2024-09-16T14:10:45.617632+0000","last_clean":"2024-09-16T14:10:45.617632+0000","last_became_active":"2024-09-16T14:10:17.555698+0000","last_became_peered":"2024-09-16T14:10:17.555698+0000","last_unstale":"2024-09-16T14:10:45.617632+0000","last_undegraded":"2024-09-16T14:10:45.617632+0000","last_fullsized":"2024-09-16T14:10:45.617632+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:03.785460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"198'897","reported_seq":6176,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.448811+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:25.385245+0000","last_became_peered":"2024-09-16T14:10:25.385245+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:00.028708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026361219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5530,"num_read_kb":25811,"num_write":916,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":731,"num_bytes_recovered":15701912,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589505+0000","last_change":"2024-09-16T14:10:16.472462+0000","last_active":"2024-09-16T14:10:45.589505+0000","last_peered":"2024-09-16T14:10:45.589505+0000","last_clean":"2024-09-16T14:10:45.589505+0000","last_became_active":"2024-09-16T14:10:16.471977+0000","last_became_peered":"2024-09-16T14:10:16.471977+0000","last_unstale":"2024-09-16T14:10:45.589505+0000","last_undegraded":"2024-09-16T14:10:45.589505+0000","last_fullsized":"2024-09-16T14:10:45.589505+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:09.028511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"196'967","reported_seq":7724,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589578+0000","last_change":"2024-09-16T14:10:29.448302+0000","last_active":"2024-09-16T14:10:45.589578+0000","last_peered":"2024-09-16T14:10:45.589578+0000","last_clean":"2024-09-16T14:10:45.589578+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:10:45.589578+0000","last_undegraded":"2024-09-16T14:10:45.589578+0000","last_fullsized":"2024-09-16T14:10:45.589578+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:58.173875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002259535,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":10392,"num_read_kb":45651,"num_write":1244,"num_write_kb":30465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617428+0000","last_change":"2024-09-16T14:10:00.102785+0000","last_active":"2024-09-16T14:10:45.617428+0000","last_peered":"2024-09-16T14:10:45.617428+0000","last_clean":"2024-09-16T14:10:45.617428+0000","last_became_active":"2024-09-16T14:10:00.102521+0000","last_became_peered":"2024-09-16T14:10:00.102521+0000","last_unstale":"2024-09-16T14:10:45.617428+0000","last_undegraded":"2024-09-16T14:10:45.617428+0000","last_fullsized":"2024-09-16T14:10:45.617428+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:17.608389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"197'630","reported_seq":3952,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616832+0000","last_change":"2024-09-16T14:10:29.574272+0000","last_active":"2024-09-16T14:10:45.616832+0000","last_peered":"2024-09-16T14:10:45.616832+0000","last_clean":"2024-09-16T14:10:45.616832+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:10:45.616832+0000","last_undegraded":"2024-09-16T14:10:45.616832+0000","last_fullsized":"2024-09-16T14:10:45.616832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001012095,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4560,"num_read_kb":16131,"num_write":597,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593236+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:10:46.593236+0000","last_peered":"2024-09-16T14:10:46.593236+0000","last_clean":"2024-09-16T14:10:46.593236+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:10:46.593236+0000","last_undegraded":"2024-09-16T14:10:46.593236+0000","last_fullsized":"2024-09-16T14:10:46.593236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"197'780","reported_seq":9812,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573395+0000","last_change":"2024-09-16T14:10:29.447893+0000","last_active":"2024-09-16T14:10:44.573395+0000","last_peered":"2024-09-16T14:10:44.573395+0000","last_clean":"2024-09-16T14:10:44.573395+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:10:44.573395+0000","last_undegraded":"2024-09-16T14:10:44.573395+0000","last_fullsized":"2024-09-16T14:10:44.573395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064281,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":11041,"num_read_kb":12619,"num_write":857,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"198'1412","reported_seq":5233,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590076+0000","last_change":"2024-09-16T14:10:29.449673+0000","last_active":"2024-09-16T14:10:45.590076+0000","last_peered":"2024-09-16T14:10:45.590076+0000","last_clean":"2024-09-16T14:10:45.590076+0000","last_became_active":"2024-09-16T14:10:19.505692+0000","last_became_peered":"2024-09-16T14:10:19.505692+0000","last_unstale":"2024-09-16T14:10:45.590076+0000","last_undegraded":"2024-09-16T14:10:45.590076+0000","last_fullsized":"2024-09-16T14:10:45.590076+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:54.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040141910000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":9687,"num_read_kb":19616,"num_write":1641,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617596+0000","last_change":"2024-09-16T14:10:00.110539+0000","last_active":"2024-09-16T14:10:45.617596+0000","last_peered":"2024-09-16T14:10:45.617596+0000","last_clean":"2024-09-16T14:10:45.617596+0000","last_became_active":"2024-09-16T14:10:00.104539+0000","last_became_peered":"2024-09-16T14:10:00.104539+0000","last_unstale":"2024-09-16T14:10:45.617596+0000","last_undegraded":"2024-09-16T14:10:45.617596+0000","last_fullsized":"2024-09-16T14:10:45.617596+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:07.278197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"196'1316","reported_seq":6546,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592682+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592682+0000","last_peered":"2024-09-16T14:10:46.592682+0000","last_clean":"2024-09-16T14:10:46.592682+0000","last_became_active":"2024-09-16T14:10:26.385527+0000","last_became_peered":"2024-09-16T14:10:26.385527+0000","last_unstale":"2024-09-16T14:10:46.592682+0000","last_undegraded":"2024-09-16T14:10:46.592682+0000","last_fullsized":"2024-09-16T14:10:46.592682+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1316,"log_dups_size":0,"ondisk_log_size":1316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:03.910845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051618139999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9488,"num_read_kb":24272,"num_write":1738,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":8428888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617802+0000","last_change":"2024-09-16T14:10:21.388025+0000","last_active":"2024-09-16T14:10:45.617802+0000","last_peered":"2024-09-16T14:10:45.617802+0000","last_clean":"2024-09-16T14:10:45.617802+0000","last_became_active":"2024-09-16T14:10:21.387173+0000","last_became_peered":"2024-09-16T14:10:21.387173+0000","last_unstale":"2024-09-16T14:10:45.617802+0000","last_undegraded":"2024-09-16T14:10:45.617802+0000","last_fullsized":"2024-09-16T14:10:45.617802+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:57.862024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"197'820","reported_seq":7030,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616505+0000","last_change":"2024-09-16T14:10:29.574667+0000","last_active":"2024-09-16T14:10:45.616505+0000","last_peered":"2024-09-16T14:10:45.616505+0000","last_clean":"2024-09-16T14:10:45.616505+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:10:45.616505+0000","last_undegraded":"2024-09-16T14:10:45.616505+0000","last_fullsized":"2024-09-16T14:10:45.616505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233734,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":8272,"num_read_kb":21183,"num_write":947,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617104+0000","last_change":"2024-09-16T14:10:00.104224+0000","last_active":"2024-09-16T14:10:45.617104+0000","last_peered":"2024-09-16T14:10:45.617104+0000","last_clean":"2024-09-16T14:10:45.617104+0000","last_became_active":"2024-09-16T14:10:00.104003+0000","last_became_peered":"2024-09-16T14:10:00.104003+0000","last_unstale":"2024-09-16T14:10:45.617104+0000","last_undegraded":"2024-09-16T14:10:45.617104+0000","last_fullsized":"2024-09-16T14:10:45.617104+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:08.523062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"197'667","reported_seq":15108,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589347+0000","last_change":"2024-09-16T14:10:29.448082+0000","last_active":"2024-09-16T14:10:45.589347+0000","last_peered":"2024-09-16T14:10:45.589347+0000","last_clean":"2024-09-16T14:10:45.589347+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:10:45.589347+0000","last_undegraded":"2024-09-16T14:10:45.589347+0000","last_fullsized":"2024-09-16T14:10:45.589347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:15.668078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020715640000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":14807,"num_read_kb":52699,"num_write":621,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617295+0000","last_change":"2024-09-16T14:10:00.112462+0000","last_active":"2024-09-16T14:10:45.617295+0000","last_peered":"2024-09-16T14:10:45.617295+0000","last_clean":"2024-09-16T14:10:45.617295+0000","last_became_active":"2024-09-16T14:10:00.105302+0000","last_became_peered":"2024-09-16T14:10:00.105302+0000","last_unstale":"2024-09-16T14:10:45.617295+0000","last_undegraded":"2024-09-16T14:10:45.617295+0000","last_fullsized":"2024-09-16T14:10:45.617295+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:34:48.094821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591766+0000","last_change":"2024-09-16T14:09:14.399362+0000","last_active":"2024-09-16T14:10:46.591766+0000","last_peered":"2024-09-16T14:10:46.591766+0000","last_clean":"2024-09-16T14:10:46.591766+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:10:46.591766+0000","last_undegraded":"2024-09-16T14:10:46.591766+0000","last_fullsized":"2024-09-16T14:10:46.591766+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:18.538987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"198'704","reported_seq":4286,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589250+0000","last_change":"2024-09-16T14:10:29.446791+0000","last_active":"2024-09-16T14:10:45.589250+0000","last_peered":"2024-09-16T14:10:45.589250+0000","last_clean":"2024-09-16T14:10:45.589250+0000","last_became_active":"2024-09-16T14:10:26.384950+0000","last_became_peered":"2024-09-16T14:10:26.384950+0000","last_unstale":"2024-09-16T14:10:45.589250+0000","last_undegraded":"2024-09-16T14:10:45.589250+0000","last_fullsized":"2024-09-16T14:10:45.589250+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:46.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088242400000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4245,"num_read_kb":11739,"num_write":722,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":2087145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"198'623","reported_seq":6962,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617420+0000","last_change":"2024-09-16T14:10:29.578958+0000","last_active":"2024-09-16T14:10:45.617420+0000","last_peered":"2024-09-16T14:10:45.617420+0000","last_clean":"2024-09-16T14:10:45.617420+0000","last_became_active":"2024-09-16T14:10:08.396086+0000","last_became_peered":"2024-09-16T14:10:08.396086+0000","last_unstale":"2024-09-16T14:10:45.617420+0000","last_undegraded":"2024-09-16T14:10:45.617420+0000","last_fullsized":"2024-09-16T14:10:45.617420+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:20.827364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00321149,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6657,"num_read_kb":16604,"num_write":570,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":10317335,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"197'850","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617685+0000","last_change":"2024-09-16T14:10:29.580121+0000","last_active":"2024-09-16T14:10:45.617685+0000","last_peered":"2024-09-16T14:10:45.617685+0000","last_clean":"2024-09-16T14:10:45.617685+0000","last_became_active":"2024-09-16T14:10:07.390964+0000","last_became_peered":"2024-09-16T14:10:07.390964+0000","last_unstale":"2024-09-16T14:10:45.617685+0000","last_undegraded":"2024-09-16T14:10:45.617685+0000","last_fullsized":"2024-09-16T14:10:45.617685+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:48.546737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041712620000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":7231,"num_read_kb":39110,"num_write":775,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":49748621,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"196'747","reported_seq":5182,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617930+0000","last_change":"2024-09-16T14:10:29.457917+0000","last_active":"2024-09-16T14:10:45.617930+0000","last_peered":"2024-09-16T14:10:45.617930+0000","last_clean":"2024-09-16T14:10:45.617930+0000","last_became_active":"2024-09-16T14:10:24.410718+0000","last_became_peered":"2024-09-16T14:10:24.410718+0000","last_unstale":"2024-09-16T14:10:45.617930+0000","last_undegraded":"2024-09-16T14:10:45.617930+0000","last_fullsized":"2024-09-16T14:10:45.617930+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:31.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064224900000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7094,"num_read_kb":44664,"num_write":801,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":19444502,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"197'621","reported_seq":8134,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590037+0000","last_change":"2024-09-16T14:10:29.448787+0000","last_active":"2024-09-16T14:10:45.590037+0000","last_peered":"2024-09-16T14:10:45.590037+0000","last_clean":"2024-09-16T14:10:45.590037+0000","last_became_active":"2024-09-16T14:10:25.384634+0000","last_became_peered":"2024-09-16T14:10:25.384634+0000","last_unstale":"2024-09-16T14:10:45.590037+0000","last_undegraded":"2024-09-16T14:10:45.590037+0000","last_fullsized":"2024-09-16T14:10:45.590037+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:34.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034031370000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7377,"num_read_kb":27693,"num_write":494,"num_write_kb":13033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":23836787,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"197'496","reported_seq":2794,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617334+0000","last_change":"2024-09-16T14:10:29.581839+0000","last_active":"2024-09-16T14:10:45.617334+0000","last_peered":"2024-09-16T14:10:45.617334+0000","last_clean":"2024-09-16T14:10:45.617334+0000","last_became_active":"2024-09-16T14:10:04.123286+0000","last_became_peered":"2024-09-16T14:10:04.123286+0000","last_unstale":"2024-09-16T14:10:45.617334+0000","last_undegraded":"2024-09-16T14:10:45.617334+0000","last_fullsized":"2024-09-16T14:10:45.617334+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:39.608895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054803270000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":25169,"num_write":396,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":20371845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"198'510","reported_seq":7398,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573799+0000","last_change":"2024-09-16T14:10:29.446896+0000","last_active":"2024-09-16T14:10:44.573799+0000","last_peered":"2024-09-16T14:10:44.573799+0000","last_clean":"2024-09-16T14:10:44.573799+0000","last_became_active":"2024-09-16T14:10:16.487713+0000","last_became_peered":"2024-09-16T14:10:16.487713+0000","last_unstale":"2024-09-16T14:10:44.573799+0000","last_undegraded":"2024-09-16T14:10:44.573799+0000","last_fullsized":"2024-09-16T14:10:44.573799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:51.235912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072767700000000003,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7823,"num_read_kb":31641,"num_write":482,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"197'1003","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590342+0000","last_change":"2024-09-16T14:10:29.450285+0000","last_active":"2024-09-16T14:10:45.590342+0000","last_peered":"2024-09-16T14:10:45.590342+0000","last_clean":"2024-09-16T14:10:45.590342+0000","last_became_active":"2024-09-16T14:10:19.501036+0000","last_became_peered":"2024-09-16T14:10:19.501036+0000","last_unstale":"2024-09-16T14:10:45.590342+0000","last_undegraded":"2024-09-16T14:10:45.590342+0000","last_fullsized":"2024-09-16T14:10:45.590342+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:10.222500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041967539999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10106,"num_read_kb":24848,"num_write":1204,"num_write_kb":15655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":8219627,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"193'625","reported_seq":12640,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591811+0000","last_change":"2024-09-16T14:10:29.462983+0000","last_active":"2024-09-16T14:10:46.591811+0000","last_peered":"2024-09-16T14:10:46.591811+0000","last_clean":"2024-09-16T14:10:46.591811+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:10:46.591811+0000","last_undegraded":"2024-09-16T14:10:46.591811+0000","last_fullsized":"2024-09-16T14:10:46.591811+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:30.500273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114802,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13307,"num_read_kb":24046,"num_write":748,"num_write_kb":16341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"196'986","reported_seq":6243,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617242+0000","last_change":"2024-09-16T14:10:29.578998+0000","last_active":"2024-09-16T14:10:45.617242+0000","last_peered":"2024-09-16T14:10:45.617242+0000","last_clean":"2024-09-16T14:10:45.617242+0000","last_became_active":"2024-09-16T14:10:08.399369+0000","last_became_peered":"2024-09-16T14:10:08.399369+0000","last_unstale":"2024-09-16T14:10:45.617242+0000","last_undegraded":"2024-09-16T14:10:45.617242+0000","last_fullsized":"2024-09-16T14:10:45.617242+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:14.789387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036375169999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6839,"num_read_kb":16656,"num_write":1300,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":7816767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"198'741","reported_seq":6825,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616511+0000","last_change":"2024-09-16T14:10:29.576917+0000","last_active":"2024-09-16T14:10:45.616511+0000","last_peered":"2024-09-16T14:10:45.616511+0000","last_clean":"2024-09-16T14:10:45.616511+0000","last_became_active":"2024-09-16T14:10:05.121914+0000","last_became_peered":"2024-09-16T14:10:05.121914+0000","last_unstale":"2024-09-16T14:10:45.616511+0000","last_undegraded":"2024-09-16T14:10:45.616511+0000","last_fullsized":"2024-09-16T14:10:45.616511+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:38.113333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025226160000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7908,"num_read_kb":56867,"num_write":705,"num_write_kb":31085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":28924045,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"193'753","reported_seq":7845,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617139+0000","last_change":"2024-09-16T14:10:29.580755+0000","last_active":"2024-09-16T14:10:45.617139+0000","last_peered":"2024-09-16T14:10:45.617139+0000","last_clean":"2024-09-16T14:10:45.617139+0000","last_became_active":"2024-09-16T14:10:21.385616+0000","last_became_peered":"2024-09-16T14:10:21.385616+0000","last_unstale":"2024-09-16T14:10:45.617139+0000","last_undegraded":"2024-09-16T14:10:45.617139+0000","last_fullsized":"2024-09-16T14:10:45.617139+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:51.854260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047871169999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8482,"num_read_kb":31842,"num_write":873,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":41601355,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":588,"num_read":281369,"num_read_kb":918178,"num_write":31085,"num_write_kb":441735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9458,"num_bytes_recovered":336276108,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37982,"internal_metadata":0},"log_size":27091,"ondisk_log_size":27091,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304052,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067788,"statfs":{"total":96636764160,"available":96325414912,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.92100000000000004,"15min":0.90900000000000003},"min":{"1min":0.54700000000000004,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.4790000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.91100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.25,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.621}]},{"osd":1,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85199999999999998,"15min":0.85999999999999999},"min":{"1min":0.41199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0409999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.341,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1439999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.52900000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2529999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.371,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.76800000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0149999999999999,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2689999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.375},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.0089999999999999},"min":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.252}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.5549999999999999,"5min":3.246,"15min":3.246},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.024,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2909999999999999,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.52}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.0249999999999999},"min":{"1min":0.63700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.165,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.042},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0189999999999999},"min":{"1min":0.47199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5229999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":1.5840000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0840000000000001,"15min":1.069},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4650000000000001,"5min":3.492,"15min":3.492},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.123,"15min":1.131},"min":{"1min":0.59899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.577,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.6819999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312352,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059488,"statfs":{"total":96636764160,"available":96316915712,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1850000000000001,"5min":1.498,"15min":1.498},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2729999999999999,"5min":1.413,"15min":1.413},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95499999999999996,"15min":0.995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.77400000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84099999999999997,"15min":0.85699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.872},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.175,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.014}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.577,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88400000000000001,"15min":0.95399999999999996},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1200000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94199999999999995,"15min":0.97199999999999998},"min":{"1min":0.48299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.56100000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.976,"5min":4.976,"15min":4.976},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.004,"5min":1.03,"15min":1.0549999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.95599999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.55000000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.4319999999999999,"5min":3.4319999999999999,"15min":3.4319999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.101,"15min":1.1419999999999999},"min":{"1min":0.57399999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304116,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067724,"statfs":{"total":96636764160,"available":96325349376,"internally_reserved":0,"allocated":2654208,"data_stored":1841193,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97299999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.256},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.78700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.079}]},{"osd":1,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.85399999999999998,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.048,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90300000000000002,"15min":0.95499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0900000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.153}]},{"osd":2,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.94199999999999995,"15min":0.92300000000000004},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1479999999999999,"5min":10.999000000000001,"15min":10.999000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":11.018000000000001,"15min":11.018000000000001},"last":1.4159999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.212,"15min":1.212},"last":1.3089999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.93799999999999994},"min":{"1min":0.60399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.601,"15min":1.601},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.96699999999999997,"15min":0.996},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.244,"5min":1.764,"15min":1.764},"last":1.2070000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.538,"15min":1.538},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.999,"15min":1.0580000000000001},"min":{"1min":0.67200000000000004,"5min":0.315,"15min":0.315},"max":{"1min":1.409,"5min":1.573,"15min":1.573},"last":1.1339999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.008,"15min":1.046},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3100000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.0660000000000001,"15min":1.1220000000000001},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3759999999999999,"5min":1.619,"15min":1.619},"last":1.46}]}]},{"osd":3,"up_from":83,"seq":356482285598,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309916,"kb_used_data":2512,"kb_used_omap":31,"kb_used_meta":307360,"kb_avail":94061924,"statfs":{"total":96636764160,"available":96319410176,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31864,"internal_metadata":314737544},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.36699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":1.468,"15min":1.468},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1459999999999999,"5min":1.258,"15min":1.258},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93200000000000005,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.89600000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.119,"5min":1.347,"15min":1.347},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.96399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.2250000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95999999999999996,"15min":0.95299999999999996},"min":{"1min":0.64900000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.1439999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0349999999999999,"15min":1.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.03},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.042},"min":{"1min":0.72099999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.454,"5min":1.486,"15min":1.486},"last":1.202}]},{"osd":7,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.024,"15min":1.024},"min":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2929999999999999,"5min":1.419,"15min":1.419},"last":1.306},{"interface":"front","average":{"1min":1.103,"5min":1.0369999999999999,"15min":1.026},"min":{"1min":0.77500000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4139999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.2729999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308184,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063656,"statfs":{"total":96636764160,"available":96321183744,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.276,"5min":1.454,"15min":1.454},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Sep 16 14:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.5,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.753}]},{"osd":4,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.878}]},{"osd":5,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4390000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.139},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.948,"15min":1.948},"last":1.1839999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92700000000000005,"15min":0.93200000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.026,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.415,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.008}]},{"osd":7,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.98799999999999999,"15min":0.999},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.623,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.022,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4470000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.0469999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312424,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059416,"statfs":{"total":96636764160,"available":96316841984,"internally_reserved":0,"allocated":2637824,"data_stored":1841193,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.80800000000000005,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.5890000000000004,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.89800000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.70299999999999996,"15min":0.745},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.90500000000000003,"5min":1.998,"15min":1.998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Mon Sep 16 14:10:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":7.9950000000000001,"5min":7.9950000000000001,"15min":7.9950000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0009999999999999,"5min":1.244,"15min":1.244},"last":0.82199999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.89200000000000002},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.89600000000000002,"15min":0.92600000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1359999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.087,"15min":1.1200000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2330000000000001,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":1.034},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.077,"15min":1.097},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0600000000000001,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0429999999999999,"15min":1.0960000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.874}]},{"osd":7,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.018,"15min":1.0649999999999999},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.865,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.125},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.145,"15min":1.149},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.375,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":1.1930000000000001}]}]},{"osd":0,"up_from":184,"seq":790273982473,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311836,"kb_used_data":2508,"kb_used_omap":125,"kb_used_meta":309186,"kb_avail":94060004,"statfs":{"total":96636764160,"available":96317444096,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":128171,"internal_metadata":316607317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316520,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94055320,"statfs":{"total":96636764160,"available":96312647680,"internally_reserved":0,"allocated":2580480,"data_stored":1245005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2809999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.268},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1990000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.2210000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.121,"5min":37.805,"15min":37.805},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1819999999999999,"15min":1.008},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.155,"5min":37.857999999999997,"15min":37.857999999999997},"last":0.91300000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.115,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.204,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1100000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:10:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.68000000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.133,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.60999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2829999999999999,"5min":1.611,"15min":1.611},"last":0.83499999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.95899999999999996,"15min":0.97299999999999998},"min":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3009999999999999,"5min":1.429,"15min":1.429},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.67700000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.278,"5min":1.601,"15min":1.601},"last":1.383}]},{"osd":7,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68700000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.355,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0780000000000001,"15min":1.097},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3819999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.236}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29034,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:10:52.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:10:52.324 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:52.328 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:10:52.343 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-16T14:10:50.830421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":588,"num_read":281435,"num_read_kb":918233,"num_write":31188,"num_write_kb":443365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9465,"num_bytes_recovered":336866533,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27166,"ondisk_log_size":27166,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2479400,"kb_used_data":20260,"kb_used_omap":166,"kb_used_meta":2458841,"kb_avail":752495320,"statfs":{"total":773094113280,"available":770555207680,"internally_reserved":0,"allocated":20746240,"data_stored":11135051,"data_compressed":11598,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":170259,"internal_metadata":2517853933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001577"},"pg_stats":[{"pgid":"3.1f","version":"198'1876","reported_seq":7887,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592881+0000","last_change":"2024-09-16T14:10:29.531167+0000","last_active":"2024-09-16T14:10:46.592881+0000","last_peered":"2024-09-16T14:10:46.592881+0000","last_clean":"2024-09-16T14:10:46.592881+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:10:46.592881+0000","last_undegraded":"2024-09-16T14:10:46.592881+0000","last_fullsized":"2024-09-16T14:10:46.592881+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1876,"log_dups_size":0,"ondisk_log_size":1876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:02.959438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039454019999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15599,"num_read_kb":19767,"num_write":2973,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"196'800","reported_seq":2726,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592990+0000","last_change":"2024-09-16T14:10:29.528090+0000","last_active":"2024-09-16T14:10:46.592990+0000","last_peered":"2024-09-16T14:10:46.592990+0000","last_clean":"2024-09-16T14:10:46.592990+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:10:46.592990+0000","last_undegraded":"2024-09-16T14:10:46.592990+0000","last_fullsized":"2024-09-16T14:10:46.592990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098036900000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3428,"num_read_kb":21398,"num_write":967,"num_write_kb":11613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"197'653","reported_seq":5530,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617422+0000","last_change":"2024-09-16T14:10:29.581527+0000","last_active":"2024-09-16T14:10:45.617422+0000","last_peered":"2024-09-16T14:10:45.617422+0000","last_clean":"2024-09-16T14:10:45.617422+0000","last_became_active":"2024-09-16T14:10:20.387629+0000","last_became_peered":"2024-09-16T14:10:20.387629+0000","last_unstale":"2024-09-16T14:10:45.617422+0000","last_undegraded":"2024-09-16T14:10:45.617422+0000","last_fullsized":"2024-09-16T14:10:45.617422+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:30.104851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051863780000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7537,"num_read_kb":24768,"num_write":780,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":26063912,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"198'1032","reported_seq":14527,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592725+0000","last_change":"2024-09-16T14:10:29.463373+0000","last_active":"2024-09-16T14:10:46.592725+0000","last_peered":"2024-09-16T14:10:46.592725+0000","last_clean":"2024-09-16T14:10:46.592725+0000","last_became_active":"2024-09-16T14:10:20.385216+0000","last_became_peered":"2024-09-16T14:10:20.385216+0000","last_unstale":"2024-09-16T14:10:46.592725+0000","last_undegraded":"2024-09-16T14:10:46.592725+0000","last_fullsized":"2024-09-16T14:10:46.592725+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:22.705027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013844,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":21,"num_read":15843,"num_read_kb":26936,"num_write":1369,"num_write_kb":11030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":6143753,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"197'738","reported_seq":8080,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.446977+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:20.387007+0000","last_became_peered":"2024-09-16T14:10:20.387007+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:41.660168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010460949999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10352,"num_read_kb":26550,"num_write":833,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"196'845","reported_seq":8027,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592011+0000","last_change":"2024-09-16T14:10:29.463712+0000","last_active":"2024-09-16T14:10:46.592011+0000","last_peered":"2024-09-16T14:10:46.592011+0000","last_clean":"2024-09-16T14:10:46.592011+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:10:46.592011+0000","last_undegraded":"2024-09-16T14:10:46.592011+0000","last_fullsized":"2024-09-16T14:10:46.592011+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:25.215853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017171580000000001,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":10449,"num_read_kb":28504,"num_write":803,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"197'630","reported_seq":2342,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589326+0000","last_change":"2024-09-16T14:10:29.447450+0000","last_active":"2024-09-16T14:10:45.589326+0000","last_peered":"2024-09-16T14:10:45.589326+0000","last_clean":"2024-09-16T14:10:45.589326+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:10:45.589326+0000","last_undegraded":"2024-09-16T14:10:45.589326+0000","last_fullsized":"2024-09-16T14:10:45.589326+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:41.369426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483112,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1924,"num_read_kb":28666,"num_write":709,"num_write_kb":21954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"198'968","reported_seq":6364,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592751+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592751+0000","last_peered":"2024-09-16T14:10:46.592751+0000","last_clean":"2024-09-16T14:10:46.592751+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:10:46.592751+0000","last_undegraded":"2024-09-16T14:10:46.592751+0000","last_fullsized":"2024-09-16T14:10:46.592751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050019210000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9238,"num_read_kb":21812,"num_write":1162,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"194'1351","reported_seq":6429,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593259+0000","last_change":"2024-09-16T14:10:29.531182+0000","last_active":"2024-09-16T14:10:46.593259+0000","last_peered":"2024-09-16T14:10:46.593259+0000","last_clean":"2024-09-16T14:10:46.593259+0000","last_became_active":"2024-09-16T14:10:05.123005+0000","last_became_peered":"2024-09-16T14:10:05.123005+0000","last_unstale":"2024-09-16T14:10:46.593259+0000","last_undegraded":"2024-09-16T14:10:46.593259+0000","last_fullsized":"2024-09-16T14:10:46.593259+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:44.294281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039366710000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8929,"num_read_kb":33272,"num_write":1987,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":4502377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"197'751","reported_seq":13941,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573574+0000","last_change":"2024-09-16T14:10:29.446978+0000","last_active":"2024-09-16T14:10:44.573574+0000","last_peered":"2024-09-16T14:10:44.573574+0000","last_clean":"2024-09-16T14:10:44.573574+0000","last_became_active":"2024-09-16T14:10:05.377635+0000","last_became_peered":"2024-09-16T14:10:05.377635+0000","last_unstale":"2024-09-16T14:10:44.573574+0000","last_undegraded":"2024-09-16T14:10:44.573574+0000","last_fullsized":"2024-09-16T14:10:44.573574+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:42.336862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089383699999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":14525,"num_read_kb":50269,"num_write":689,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":11596768,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"198'576","reported_seq":5189,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589895+0000","last_change":"2024-09-16T14:10:29.448044+0000","last_active":"2024-09-16T14:10:45.589895+0000","last_peered":"2024-09-16T14:10:45.589895+0000","last_clean":"2024-09-16T14:10:45.589895+0000","last_became_active":"2024-09-16T14:10:19.498121+0000","last_became_peered":"2024-09-16T14:10:19.498121+0000","last_unstale":"2024-09-16T14:10:45.589895+0000","last_undegraded":"2024-09-16T14:10:45.589895+0000","last_fullsized":"2024-09-16T14:10:45.589895+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:08.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002716544,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4971,"num_read_kb":28564,"num_write":500,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":50024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"197'723","reported_seq":7824,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617501+0000","last_change":"2024-09-16T14:10:29.582367+0000","last_active":"2024-09-16T14:10:45.617501+0000","last_peered":"2024-09-16T14:10:45.617501+0000","last_clean":"2024-09-16T14:10:45.617501+0000","last_became_active":"2024-09-16T14:10:06.388650+0000","last_became_peered":"2024-09-16T14:10:06.388650+0000","last_unstale":"2024-09-16T14:10:45.617501+0000","last_undegraded":"2024-09-16T14:10:45.617501+0000","last_fullsized":"2024-09-16T14:10:45.617501+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:30.578665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054744670000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9236,"num_read_kb":38811,"num_write":682,"num_write_kb":10309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":10174643,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617632+0000","last_change":"2024-09-16T14:10:17.556015+0000","last_active":"2024-09-16T14:10:45.617632+0000","last_peered":"2024-09-16T14:10:45.617632+0000","last_clean":"2024-09-16T14:10:45.617632+0000","last_became_active":"2024-09-16T14:10:17.555698+0000","last_became_peered":"2024-09-16T14:10:17.555698+0000","last_unstale":"2024-09-16T14:10:45.617632+0000","last_undegraded":"2024-09-16T14:10:45.617632+0000","last_fullsized":"2024-09-16T14:10:45.617632+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:03.785460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"198'897","reported_seq":6176,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.448811+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:25.385245+0000","last_became_peered":"2024-09-16T14:10:25.385245+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:00.028708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026361219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5530,"num_read_kb":25811,"num_write":916,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":731,"num_bytes_recovered":15701912,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589505+0000","last_change":"2024-09-16T14:10:16.472462+0000","last_active":"2024-09-16T14:10:45.589505+0000","last_peered":"2024-09-16T14:10:45.589505+0000","last_clean":"2024-09-16T14:10:45.589505+0000","last_became_active":"2024-09-16T14:10:16.471977+0000","last_became_peered":"2024-09-16T14:10:16.471977+0000","last_unstale":"2024-09-16T14:10:45.589505+0000","last_undegraded":"2024-09-16T14:10:45.589505+0000","last_fullsized":"2024-09-16T14:10:45.589505+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:09.028511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"196'967","reported_seq":7724,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589578+0000","last_change":"2024-09-16T14:10:29.448302+0000","last_active":"2024-09-16T14:10:45.589578+0000","last_peered":"2024-09-16T14:10:45.589578+0000","last_clean":"2024-09-16T14:10:45.589578+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:10:45.589578+0000","last_undegraded":"2024-09-16T14:10:45.589578+0000","last_fullsized":"2024-09-16T14:10:45.589578+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:58.173875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002259535,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":10392,"num_read_kb":45651,"num_write":1244,"num_write_kb":30465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617428+0000","last_change":"2024-09-16T14:10:00.102785+0000","last_active":"2024-09-16T14:10:45.617428+0000","last_peered":"2024-09-16T14:10:45.617428+0000","last_clean":"2024-09-16T14:10:45.617428+0000","last_became_active":"2024-09-16T14:10:00.102521+0000","last_became_peered":"2024-09-16T14:10:00.102521+0000","last_unstale":"2024-09-16T14:10:45.617428+0000","last_undegraded":"2024-09-16T14:10:45.617428+0000","last_fullsized":"2024-09-16T14:10:45.617428+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:17.608389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"197'630","reported_seq":3952,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616832+0000","last_change":"2024-09-16T14:10:29.574272+0000","last_active":"2024-09-16T14:10:45.616832+0000","last_peered":"2024-09-16T14:10:45.616832+0000","last_clean":"2024-09-16T14:10:45.616832+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:10:45.616832+0000","last_undegraded":"2024-09-16T14:10:45.616832+0000","last_fullsized":"2024-09-16T14:10:45.616832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001012095,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4560,"num_read_kb":16131,"num_write":597,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593236+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:10:46.593236+0000","last_peered":"2024-09-16T14:10:46.593236+0000","last_clean":"2024-09-16T14:10:46.593236+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:10:46.593236+0000","last_undegraded":"2024-09-16T14:10:46.593236+0000","last_fullsized":"2024-09-16T14:10:46.593236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"197'780","reported_seq":9812,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573395+0000","last_change":"2024-09-16T14:10:29.447893+0000","last_active":"2024-09-16T14:10:44.573395+0000","last_peered":"2024-09-16T14:10:44.573395+0000","last_clean":"2024-09-16T14:10:44.573395+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:10:44.573395+0000","last_undegraded":"2024-09-16T14:10:44.573395+0000","last_fullsized":"2024-09-16T14:10:44.573395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064281,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":11041,"num_read_kb":12619,"num_write":857,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"198'1412","reported_seq":5233,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590076+0000","last_change":"2024-09-16T14:10:29.449673+0000","last_active":"2024-09-16T14:10:45.590076+0000","last_peered":"2024-09-16T14:10:45.590076+0000","last_clean":"2024-09-16T14:10:45.590076+0000","last_became_active":"2024-09-16T14:10:19.505692+0000","last_became_peered":"2024-09-16T14:10:19.505692+0000","last_unstale":"2024-09-16T14:10:45.590076+0000","last_undegraded":"2024-09-16T14:10:45.590076+0000","last_fullsized":"2024-09-16T14:10:45.590076+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:54.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040141910000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":9687,"num_read_kb":19616,"num_write":1641,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617596+0000","last_change":"2024-09-16T14:10:00.110539+0000","last_active":"2024-09-16T14:10:45.617596+0000","last_peered":"2024-09-16T14:10:45.617596+0000","last_clean":"2024-09-16T14:10:45.617596+0000","last_became_active":"2024-09-16T14:10:00.104539+0000","last_became_peered":"2024-09-16T14:10:00.104539+0000","last_unstale":"2024-09-16T14:10:45.617596+0000","last_undegraded":"2024-09-16T14:10:45.617596+0000","last_fullsized":"2024-09-16T14:10:45.617596+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:07.278197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"196'1316","reported_seq":6546,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592682+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592682+0000","last_peered":"2024-09-16T14:10:46.592682+0000","last_clean":"2024-09-16T14:10:46.592682+0000","last_became_active":"2024-09-16T14:10:26.385527+0000","last_became_peered":"2024-09-16T14:10:26.385527+0000","last_unstale":"2024-09-16T14:10:46.592682+0000","last_undegraded":"2024-09-16T14:10:46.592682+0000","last_fullsized":"2024-09-16T14:10:46.592682+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1316,"log_dups_size":0,"ondisk_log_size":1316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:03.910845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051618139999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9488,"num_read_kb":24272,"num_write":1738,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":8428888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617802+0000","last_change":"2024-09-16T14:10:21.388025+0000","last_active":"2024-09-16T14:10:45.617802+0000","last_peered":"2024-09-16T14:10:45.617802+0000","last_clean":"2024-09-16T14:10:45.617802+0000","last_became_active":"2024-09-16T14:10:21.387173+0000","last_became_peered":"2024-09-16T14:10:21.387173+0000","last_unstale":"2024-09-16T14:10:45.617802+0000","last_undegraded":"2024-09-16T14:10:45.617802+0000","last_fullsized":"2024-09-16T14:10:45.617802+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:57.862024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"197'820","reported_seq":7030,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616505+0000","last_change":"2024-09-16T14:10:29.574667+0000","last_active":"2024-09-16T14:10:45.616505+0000","last_peered":"2024-09-16T14:10:45.616505+0000","last_clean":"2024-09-16T14:10:45.616505+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:10:45.616505+0000","last_undegraded":"2024-09-16T14:10:45.616505+0000","last_fullsized":"2024-09-16T14:10:45.616505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233734,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":8272,"num_read_kb":21183,"num_write":947,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617104+0000","last_change":"2024-09-16T14:10:00.104224+0000","last_active":"2024-09-16T14:10:45.617104+0000","last_peered":"2024-09-16T14:10:45.617104+0000","last_clean":"2024-09-16T14:10:45.617104+0000","last_became_active":"2024-09-16T14:10:00.104003+0000","last_became_peered":"2024-09-16T14:10:00.104003+0000","last_unstale":"2024-09-16T14:10:45.617104+0000","last_undegraded":"2024-09-16T14:10:45.617104+0000","last_fullsized":"2024-09-16T14:10:45.617104+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:08.523062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"197'667","reported_seq":15108,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589347+0000","last_change":"2024-09-16T14:10:29.448082+0000","last_active":"2024-09-16T14:10:45.589347+0000","last_peered":"2024-09-16T14:10:45.589347+0000","last_clean":"2024-09-16T14:10:45.589347+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:10:45.589347+0000","last_undegraded":"2024-09-16T14:10:45.589347+0000","last_fullsized":"2024-09-16T14:10:45.589347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:15.668078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020715640000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":14807,"num_read_kb":52699,"num_write":621,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617295+0000","last_change":"2024-09-16T14:10:00.112462+0000","last_active":"2024-09-16T14:10:45.617295+0000","last_peered":"2024-09-16T14:10:45.617295+0000","last_clean":"2024-09-16T14:10:45.617295+0000","last_became_active":"2024-09-16T14:10:00.105302+0000","last_became_peered":"2024-09-16T14:10:00.105302+0000","last_unstale":"2024-09-16T14:10:45.617295+0000","last_undegraded":"2024-09-16T14:10:45.617295+0000","last_fullsized":"2024-09-16T14:10:45.617295+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:34:48.094821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591766+0000","last_change":"2024-09-16T14:09:14.399362+0000","last_active":"2024-09-16T14:10:46.591766+0000","last_peered":"2024-09-16T14:10:46.591766+0000","last_clean":"2024-09-16T14:10:46.591766+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:10:46.591766+0000","last_undegraded":"2024-09-16T14:10:46.591766+0000","last_fullsized":"2024-09-16T14:10:46.591766+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:18.538987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"198'704","reported_seq":4286,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589250+0000","last_change":"2024-09-16T14:10:29.446791+0000","last_active":"2024-09-16T14:10:45.589250+0000","last_peered":"2024-09-16T14:10:45.589250+0000","last_clean":"2024-09-16T14:10:45.589250+0000","last_became_active":"2024-09-16T14:10:26.384950+0000","last_became_peered":"2024-09-16T14:10:26.384950+0000","last_unstale":"2024-09-16T14:10:45.589250+0000","last_undegraded":"2024-09-16T14:10:45.589250+0000","last_fullsized":"2024-09-16T14:10:45.589250+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:46.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088242400000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4245,"num_read_kb":11739,"num_write":722,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":2087145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"198'623","reported_seq":6962,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617420+0000","last_change":"2024-09-16T14:10:29.578958+0000","last_active":"2024-09-16T14:10:45.617420+0000","last_peered":"2024-09-16T14:10:45.617420+0000","last_clean":"2024-09-16T14:10:45.617420+0000","last_became_active":"2024-09-16T14:10:08.396086+0000","last_became_peered":"2024-09-16T14:10:08.396086+0000","last_unstale":"2024-09-16T14:10:45.617420+0000","last_undegraded":"2024-09-16T14:10:45.617420+0000","last_fullsized":"2024-09-16T14:10:45.617420+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:20.827364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00321149,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6657,"num_read_kb":16604,"num_write":570,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":10317335,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"197'850","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617685+0000","last_change":"2024-09-16T14:10:29.580121+0000","last_active":"2024-09-16T14:10:45.617685+0000","last_peered":"2024-09-16T14:10:45.617685+0000","last_clean":"2024-09-16T14:10:45.617685+0000","last_became_active":"2024-09-16T14:10:07.390964+0000","last_became_peered":"2024-09-16T14:10:07.390964+0000","last_unstale":"2024-09-16T14:10:45.617685+0000","last_undegraded":"2024-09-16T14:10:45.617685+0000","last_fullsized":"2024-09-16T14:10:45.617685+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:48.546737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041712620000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":7231,"num_read_kb":39110,"num_write":775,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":49748621,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"196'747","reported_seq":5182,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617930+0000","last_change":"2024-09-16T14:10:29.457917+0000","last_active":"2024-09-16T14:10:45.617930+0000","last_peered":"2024-09-16T14:10:45.617930+0000","last_clean":"2024-09-16T14:10:45.617930+0000","last_became_active":"2024-09-16T14:10:24.410718+0000","last_became_peered":"2024-09-16T14:10:24.410718+0000","last_unstale":"2024-09-16T14:10:45.617930+0000","last_undegraded":"2024-09-16T14:10:45.617930+0000","last_fullsized":"2024-09-16T14:10:45.617930+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:31.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064224900000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7094,"num_read_kb":44664,"num_write":801,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":19444502,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"197'621","reported_seq":8134,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590037+0000","last_change":"2024-09-16T14:10:29.448787+0000","last_active":"2024-09-16T14:10:45.590037+0000","last_peered":"2024-09-16T14:10:45.590037+0000","last_clean":"2024-09-16T14:10:45.590037+0000","last_became_active":"2024-09-16T14:10:25.384634+0000","last_became_peered":"2024-09-16T14:10:25.384634+0000","last_unstale":"2024-09-16T14:10:45.590037+0000","last_undegraded":"2024-09-16T14:10:45.590037+0000","last_fullsized":"2024-09-16T14:10:45.590037+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:34.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034031370000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7377,"num_read_kb":27693,"num_write":494,"num_write_kb":13033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":23836787,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"197'496","reported_seq":2794,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617334+0000","last_change":"2024-09-16T14:10:29.581839+0000","last_active":"2024-09-16T14:10:45.617334+0000","last_peered":"2024-09-16T14:10:45.617334+0000","last_clean":"2024-09-16T14:10:45.617334+0000","last_became_active":"2024-09-16T14:10:04.123286+0000","last_became_peered":"2024-09-16T14:10:04.123286+0000","last_unstale":"2024-09-16T14:10:45.617334+0000","last_undegraded":"2024-09-16T14:10:45.617334+0000","last_fullsized":"2024-09-16T14:10:45.617334+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:39.608895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054803270000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":25169,"num_write":396,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":20371845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"198'510","reported_seq":7398,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573799+0000","last_change":"2024-09-16T14:10:29.446896+0000","last_active":"2024-09-16T14:10:44.573799+0000","last_peered":"2024-09-16T14:10:44.573799+0000","last_clean":"2024-09-16T14:10:44.573799+0000","last_became_active":"2024-09-16T14:10:16.487713+0000","last_became_peered":"2024-09-16T14:10:16.487713+0000","last_unstale":"2024-09-16T14:10:44.573799+0000","last_undegraded":"2024-09-16T14:10:44.573799+0000","last_fullsized":"2024-09-16T14:10:44.573799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:51.235912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072767700000000003,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7823,"num_read_kb":31641,"num_write":482,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"197'1003","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590342+0000","last_change":"2024-09-16T14:10:29.450285+0000","last_active":"2024-09-16T14:10:45.590342+0000","last_peered":"2024-09-16T14:10:45.590342+0000","last_clean":"2024-09-16T14:10:45.590342+0000","last_became_active":"2024-09-16T14:10:19.501036+0000","last_became_peered":"2024-09-16T14:10:19.501036+0000","last_unstale":"2024-09-16T14:10:45.590342+0000","last_undegraded":"2024-09-16T14:10:45.590342+0000","last_fullsized":"2024-09-16T14:10:45.590342+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:10.222500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041967539999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10106,"num_read_kb":24848,"num_write":1204,"num_write_kb":15655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":8219627,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"193'625","reported_seq":12640,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591811+0000","last_change":"2024-09-16T14:10:29.462983+0000","last_active":"2024-09-16T14:10:46.591811+0000","last_peered":"2024-09-16T14:10:46.591811+0000","last_clean":"2024-09-16T14:10:46.591811+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:10:46.591811+0000","last_undegraded":"2024-09-16T14:10:46.591811+0000","last_fullsized":"2024-09-16T14:10:46.591811+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:30.500273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114802,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13307,"num_read_kb":24046,"num_write":748,"num_write_kb":16341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"196'986","reported_seq":6243,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617242+0000","last_change":"2024-09-16T14:10:29.578998+0000","last_active":"2024-09-16T14:10:45.617242+0000","last_peered":"2024-09-16T14:10:45.617242+0000","last_clean":"2024-09-16T14:10:45.617242+0000","last_became_active":"2024-09-16T14:10:08.399369+0000","last_became_peered":"2024-09-16T14:10:08.399369+0000","last_unstale":"2024-09-16T14:10:45.617242+0000","last_undegraded":"2024-09-16T14:10:45.617242+0000","last_fullsized":"2024-09-16T14:10:45.617242+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:14.789387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036375169999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6839,"num_read_kb":16656,"num_write":1300,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":7816767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"198'741","reported_seq":6825,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616511+0000","last_change":"2024-09-16T14:10:29.576917+0000","last_active":"2024-09-16T14:10:45.616511+0000","last_peered":"2024-09-16T14:10:45.616511+0000","last_clean":"2024-09-16T14:10:45.616511+0000","last_became_active":"2024-09-16T14:10:05.121914+0000","last_became_peered":"2024-09-16T14:10:05.121914+0000","last_unstale":"2024-09-16T14:10:45.616511+0000","last_undegraded":"2024-09-16T14:10:45.616511+0000","last_fullsized":"2024-09-16T14:10:45.616511+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:38.113333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025226160000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7908,"num_read_kb":56867,"num_write":705,"num_write_kb":31085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":28924045,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"193'753","reported_seq":7845,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617139+0000","last_change":"2024-09-16T14:10:29.580755+0000","last_active":"2024-09-16T14:10:45.617139+0000","last_peered":"2024-09-16T14:10:45.617139+0000","last_clean":"2024-09-16T14:10:45.617139+0000","last_became_active":"2024-09-16T14:10:21.385616+0000","last_became_peered":"2024-09-16T14:10:21.385616+0000","last_unstale":"2024-09-16T14:10:45.617139+0000","last_undegraded":"2024-09-16T14:10:45.617139+0000","last_fullsized":"2024-09-16T14:10:45.617139+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:51.854260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047871169999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8482,"num_read_kb":31842,"num_write":873,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":41601355,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":588,"num_read":281369,"num_read_kb":918178,"num_write":31085,"num_write_kb":441735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9458,"num_bytes_recovered":336276108,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37982,"internal_metadata":0},"log_size":27091,"ondisk_log_size":27091,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304052,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067788,"statfs":{"total":96636764160,"available":96325414912,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.92100000000000004,"15min":0.90900000000000003},"min":{"1min":0.54700000000000004,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.4790000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.91100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.25,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.621}]},{"osd":1,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85199999999999998,"15min":0.85999999999999999},"min":{"1min":0.41199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0409999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.341,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1439999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.52900000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2529999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.371,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.76800000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0149999999999999,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2689999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.375},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.0089999999999999},"min":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.252}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.5549999999999999,"5min":3.246,"15min":3.246},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.024,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2909999999999999,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.52}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.0249999999999999},"min":{"1min":0.63700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.165,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.042},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0189999999999999},"min":{"1min":0.47199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5229999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":1.5840000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0840000000000001,"15min":1.069},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4650000000000001,"5min":3.492,"15min":3.492},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.123,"15min":1.131},"min":{"1min":0.59899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.577,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.6819999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312352,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059488,"statfs":{"total":96636764160,"available":96316915712,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1850000000000001,"5min":1.498,"15min":1.498},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2729999999999999,"5min":1.413,"15min":1.413},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95499999999999996,"15min":0.995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.77400000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84099999999999997,"15min":0.85699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.872},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.175,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.014}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.577,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88400000000000001,"15min":0.95399999999999996},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1200000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94199999999999995,"15min":0.97199999999999998},"min":{"1min":0.48299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.56100000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.976,"5min":4.976,"15min":4.976},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.004,"5min":1.03,"15min":1.0549999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.95599999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.55000000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.4319999999999999,"5min":3.4319999999999999,"15min":3.4319999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.101,"15min":1.1419999999999999},"min":{"1min":0.57399999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304116,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067724,"statfs":{"total":96636764160,"available":96325349376,"internally_reserved":0,"allocated":2654208,"data_stored":1841193,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97299999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.256},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.78700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.079}]},{"osd":1,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.85399999999999998,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.048,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90300000000000002,"15min":0.95499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0900000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.153}]},{"osd":2,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.94199999999999995,"15min":0.92300000000000004},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1479999999999999,"5min":10.999000000000001,"15min":10.999000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":11.018000000000001,"15min":11.018000000000001},"last":1.4159999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.212,"15min":1.212},"last":1.3089999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.93799999999999994},"min":{"1min":0.60399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.601,"15min":1.601},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.96699999999999997,"15min":0.996},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.244,"5min":1.764,"15min":1.764},"last":1.2070000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.538,"15min":1.538},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.999,"15min":1.0580000000000001},"min":{"1min":0.67200000000000004,"5min":0.315,"15min":0.315},"max":{"1min":1.409,"5min":1.573,"15min":1.573},"last":1.1339999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.008,"15min":1.046},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3100000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.0660000000000001,"15min":1.1220000000000001},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3759999999999999,"5min":1.619,"15min":1.619},"last":1.46}]}]},{"osd":3,"up_from":83,"seq":356482285598,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309916,"kb_used_data":2512,"kb_used_omap":31,"kb_used_meta":307360,"kb_avail":94061924,"statfs":{"total":96636764160,"available":96319410176,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31864,"internal_metadata":314737544},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.36699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":1.468,"15min":1.468},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1459999999999999,"5min":1.258,"15min":1.258},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93200000000000005,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.89600000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.119,"5min":1.347,"15min":1.347},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.96399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.2250000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95999999999999996,"15min":0.95299999999999996},"min":{"1min":0.64900000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.1439999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0349999999999999,"15min":1.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.03},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.042},"min":{"1min":0.72099999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.454,"5min":1.486,"15min":1.486},"last":1.202}]},{"osd":7,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.024,"15min":1.024},"min":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2929999999999999,"5min":1.419,"15min":1.419},"last":1.306},{"interface":"front","average":{"1min":1.103,"5min":1.0369999999999999,"15min":1.026},"min":{"1min":0.77500000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4139999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.2729999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308184,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063656,"statfs":{"total":96636764160,"available":96321183744,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.276,"5min":1.454,"15min":1.454},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Sep 16 14:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.5,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.753}]},{"osd":4,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.878}]},{"osd":5,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4390000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.139},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.948,"15min":1.948},"last":1.1839999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92700000000000005,"15min":0.93200000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.026,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.415,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.008}]},{"osd":7,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.98799999999999999,"15min":0.999},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.623,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.022,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4470000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.0469999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312424,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059416,"statfs":{"total":96636764160,"available":96316841984,"internally_reserved":0,"allocated":2637824,"data_stored":1841193,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.80800000000000005,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.5890000000000004,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.89800000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.70299999999999996,"15min":0.745},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.90500000000000003,"5min":1.998,"15min":1.998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Mon Sep 16 14:10:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":7.9950000000000001,"5min":7.9950000000000001,"15min":7.9950000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0009999999999999,"5min":1.244,"15min":1.244},"last":0.82199999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.89200000000000002},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.89600000000000002,"15min":0.92600000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1359999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.087,"15min":1.1200000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2330000000000001,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":1.034},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.077,"15min":1.097},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0600000000000001,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0429999999999999,"15min":1.0960000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.874}]},{"osd":7,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.018,"15min":1.0649999999999999},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.865,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.125},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.145,"15min":1.149},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.375,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":1.1930000000000001}]}]},{"osd":0,"up_from":184,"seq":790273982473,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311836,"kb_used_data":2508,"kb_used_omap":125,"kb_used_meta":309186,"kb_avail":94060004,"statfs":{"total":96636764160,"available":96317444096,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":128171,"internal_metadata":316607317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316520,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94055320,"statfs":{"total":96636764160,"available":96312647680,"internally_reserved":0,"allocated":2580480,"data_stored":1245005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2809999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.268},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1990000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.2210000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.121,"5min":37.805,"15min":37.805},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1819999999999999,"15min":1.008},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.155,"5min":37.857999999999997,"15min":37.857999999999997},"last":0.91300000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.115,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.204,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1100000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:10:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.68000000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.133,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.60999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2829999999999999,"5min":1.611,"15min":1.611},"last":0.83499999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.95899999999999996,"15min":0.97299999999999998},"min":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3009999999999999,"5min":1.429,"15min":1.429},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.67700000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.278,"5min":1.601,"15min":1.601},"last":1.383}]},{"osd":7,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68700000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.355,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0780000000000001,"15min":1.097},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3819999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.236}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29034,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:10:52.346 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T14:10:52.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:10:52.645 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:52.647 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:10:52.659 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-16T14:10:50.830421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":588,"num_read":281435,"num_read_kb":918233,"num_write":31188,"num_write_kb":443365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9465,"num_bytes_recovered":336866533,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27166,"ondisk_log_size":27166,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2479400,"kb_used_data":20260,"kb_used_omap":166,"kb_used_meta":2458841,"kb_avail":752495320,"statfs":{"total":773094113280,"available":770555207680,"internally_reserved":0,"allocated":20746240,"data_stored":11135051,"data_compressed":11598,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":170259,"internal_metadata":2517853933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001577"},"pg_stats":[{"pgid":"3.1f","version":"198'1876","reported_seq":7887,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592881+0000","last_change":"2024-09-16T14:10:29.531167+0000","last_active":"2024-09-16T14:10:46.592881+0000","last_peered":"2024-09-16T14:10:46.592881+0000","last_clean":"2024-09-16T14:10:46.592881+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:10:46.592881+0000","last_undegraded":"2024-09-16T14:10:46.592881+0000","last_fullsized":"2024-09-16T14:10:46.592881+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1876,"log_dups_size":0,"ondisk_log_size":1876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:02.959438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039454019999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15599,"num_read_kb":19767,"num_write":2973,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"196'800","reported_seq":2726,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592990+0000","last_change":"2024-09-16T14:10:29.528090+0000","last_active":"2024-09-16T14:10:46.592990+0000","last_peered":"2024-09-16T14:10:46.592990+0000","last_clean":"2024-09-16T14:10:46.592990+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:10:46.592990+0000","last_undegraded":"2024-09-16T14:10:46.592990+0000","last_fullsized":"2024-09-16T14:10:46.592990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098036900000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3428,"num_read_kb":21398,"num_write":967,"num_write_kb":11613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"197'653","reported_seq":5530,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617422+0000","last_change":"2024-09-16T14:10:29.581527+0000","last_active":"2024-09-16T14:10:45.617422+0000","last_peered":"2024-09-16T14:10:45.617422+0000","last_clean":"2024-09-16T14:10:45.617422+0000","last_became_active":"2024-09-16T14:10:20.387629+0000","last_became_peered":"2024-09-16T14:10:20.387629+0000","last_unstale":"2024-09-16T14:10:45.617422+0000","last_undegraded":"2024-09-16T14:10:45.617422+0000","last_fullsized":"2024-09-16T14:10:45.617422+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:30.104851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051863780000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7537,"num_read_kb":24768,"num_write":780,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":26063912,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"198'1032","reported_seq":14527,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592725+0000","last_change":"2024-09-16T14:10:29.463373+0000","last_active":"2024-09-16T14:10:46.592725+0000","last_peered":"2024-09-16T14:10:46.592725+0000","last_clean":"2024-09-16T14:10:46.592725+0000","last_became_active":"2024-09-16T14:10:20.385216+0000","last_became_peered":"2024-09-16T14:10:20.385216+0000","last_unstale":"2024-09-16T14:10:46.592725+0000","last_undegraded":"2024-09-16T14:10:46.592725+0000","last_fullsized":"2024-09-16T14:10:46.592725+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:22.705027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013844,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":21,"num_read":15843,"num_read_kb":26936,"num_write":1369,"num_write_kb":11030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":6143753,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"197'738","reported_seq":8080,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.446977+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:20.387007+0000","last_became_peered":"2024-09-16T14:10:20.387007+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:41.660168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010460949999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10352,"num_read_kb":26550,"num_write":833,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"196'845","reported_seq":8027,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592011+0000","last_change":"2024-09-16T14:10:29.463712+0000","last_active":"2024-09-16T14:10:46.592011+0000","last_peered":"2024-09-16T14:10:46.592011+0000","last_clean":"2024-09-16T14:10:46.592011+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:10:46.592011+0000","last_undegraded":"2024-09-16T14:10:46.592011+0000","last_fullsized":"2024-09-16T14:10:46.592011+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:25.215853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017171580000000001,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":10449,"num_read_kb":28504,"num_write":803,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"197'630","reported_seq":2342,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589326+0000","last_change":"2024-09-16T14:10:29.447450+0000","last_active":"2024-09-16T14:10:45.589326+0000","last_peered":"2024-09-16T14:10:45.589326+0000","last_clean":"2024-09-16T14:10:45.589326+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:10:45.589326+0000","last_undegraded":"2024-09-16T14:10:45.589326+0000","last_fullsized":"2024-09-16T14:10:45.589326+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:41.369426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483112,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1924,"num_read_kb":28666,"num_write":709,"num_write_kb":21954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"198'968","reported_seq":6364,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592751+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592751+0000","last_peered":"2024-09-16T14:10:46.592751+0000","last_clean":"2024-09-16T14:10:46.592751+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:10:46.592751+0000","last_undegraded":"2024-09-16T14:10:46.592751+0000","last_fullsized":"2024-09-16T14:10:46.592751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050019210000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9238,"num_read_kb":21812,"num_write":1162,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"194'1351","reported_seq":6429,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593259+0000","last_change":"2024-09-16T14:10:29.531182+0000","last_active":"2024-09-16T14:10:46.593259+0000","last_peered":"2024-09-16T14:10:46.593259+0000","last_clean":"2024-09-16T14:10:46.593259+0000","last_became_active":"2024-09-16T14:10:05.123005+0000","last_became_peered":"2024-09-16T14:10:05.123005+0000","last_unstale":"2024-09-16T14:10:46.593259+0000","last_undegraded":"2024-09-16T14:10:46.593259+0000","last_fullsized":"2024-09-16T14:10:46.593259+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:44.294281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039366710000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8929,"num_read_kb":33272,"num_write":1987,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":4502377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"197'751","reported_seq":13941,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573574+0000","last_change":"2024-09-16T14:10:29.446978+0000","last_active":"2024-09-16T14:10:44.573574+0000","last_peered":"2024-09-16T14:10:44.573574+0000","last_clean":"2024-09-16T14:10:44.573574+0000","last_became_active":"2024-09-16T14:10:05.377635+0000","last_became_peered":"2024-09-16T14:10:05.377635+0000","last_unstale":"2024-09-16T14:10:44.573574+0000","last_undegraded":"2024-09-16T14:10:44.573574+0000","last_fullsized":"2024-09-16T14:10:44.573574+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:42.336862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089383699999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":14525,"num_read_kb":50269,"num_write":689,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":11596768,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"198'576","reported_seq":5189,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589895+0000","last_change":"2024-09-16T14:10:29.448044+0000","last_active":"2024-09-16T14:10:45.589895+0000","last_peered":"2024-09-16T14:10:45.589895+0000","last_clean":"2024-09-16T14:10:45.589895+0000","last_became_active":"2024-09-16T14:10:19.498121+0000","last_became_peered":"2024-09-16T14:10:19.498121+0000","last_unstale":"2024-09-16T14:10:45.589895+0000","last_undegraded":"2024-09-16T14:10:45.589895+0000","last_fullsized":"2024-09-16T14:10:45.589895+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:08.049477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002716544,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4971,"num_read_kb":28564,"num_write":500,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":50024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"197'723","reported_seq":7824,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617501+0000","last_change":"2024-09-16T14:10:29.582367+0000","last_active":"2024-09-16T14:10:45.617501+0000","last_peered":"2024-09-16T14:10:45.617501+0000","last_clean":"2024-09-16T14:10:45.617501+0000","last_became_active":"2024-09-16T14:10:06.388650+0000","last_became_peered":"2024-09-16T14:10:06.388650+0000","last_unstale":"2024-09-16T14:10:45.617501+0000","last_undegraded":"2024-09-16T14:10:45.617501+0000","last_fullsized":"2024-09-16T14:10:45.617501+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:30.578665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054744670000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9236,"num_read_kb":38811,"num_write":682,"num_write_kb":10309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":10174643,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617632+0000","last_change":"2024-09-16T14:10:17.556015+0000","last_active":"2024-09-16T14:10:45.617632+0000","last_peered":"2024-09-16T14:10:45.617632+0000","last_clean":"2024-09-16T14:10:45.617632+0000","last_became_active":"2024-09-16T14:10:17.555698+0000","last_became_peered":"2024-09-16T14:10:17.555698+0000","last_unstale":"2024-09-16T14:10:45.617632+0000","last_undegraded":"2024-09-16T14:10:45.617632+0000","last_fullsized":"2024-09-16T14:10:45.617632+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:03.785460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"198'897","reported_seq":6176,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589345+0000","last_change":"2024-09-16T14:10:29.448811+0000","last_active":"2024-09-16T14:10:45.589345+0000","last_peered":"2024-09-16T14:10:45.589345+0000","last_clean":"2024-09-16T14:10:45.589345+0000","last_became_active":"2024-09-16T14:10:25.385245+0000","last_became_peered":"2024-09-16T14:10:25.385245+0000","last_unstale":"2024-09-16T14:10:45.589345+0000","last_undegraded":"2024-09-16T14:10:45.589345+0000","last_fullsized":"2024-09-16T14:10:45.589345+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:00.028708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026361219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5530,"num_read_kb":25811,"num_write":916,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":731,"num_bytes_recovered":15701912,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589505+0000","last_change":"2024-09-16T14:10:16.472462+0000","last_active":"2024-09-16T14:10:45.589505+0000","last_peered":"2024-09-16T14:10:45.589505+0000","last_clean":"2024-09-16T14:10:45.589505+0000","last_became_active":"2024-09-16T14:10:16.471977+0000","last_became_peered":"2024-09-16T14:10:16.471977+0000","last_unstale":"2024-09-16T14:10:45.589505+0000","last_undegraded":"2024-09-16T14:10:45.589505+0000","last_fullsized":"2024-09-16T14:10:45.589505+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:09.028511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"196'967","reported_seq":7724,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589578+0000","last_change":"2024-09-16T14:10:29.448302+0000","last_active":"2024-09-16T14:10:45.589578+0000","last_peered":"2024-09-16T14:10:45.589578+0000","last_clean":"2024-09-16T14:10:45.589578+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:10:45.589578+0000","last_undegraded":"2024-09-16T14:10:45.589578+0000","last_fullsized":"2024-09-16T14:10:45.589578+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:58.173875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002259535,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":10392,"num_read_kb":45651,"num_write":1244,"num_write_kb":30465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617428+0000","last_change":"2024-09-16T14:10:00.102785+0000","last_active":"2024-09-16T14:10:45.617428+0000","last_peered":"2024-09-16T14:10:45.617428+0000","last_clean":"2024-09-16T14:10:45.617428+0000","last_became_active":"2024-09-16T14:10:00.102521+0000","last_became_peered":"2024-09-16T14:10:00.102521+0000","last_unstale":"2024-09-16T14:10:45.617428+0000","last_undegraded":"2024-09-16T14:10:45.617428+0000","last_fullsized":"2024-09-16T14:10:45.617428+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:17.608389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"197'630","reported_seq":3952,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616832+0000","last_change":"2024-09-16T14:10:29.574272+0000","last_active":"2024-09-16T14:10:45.616832+0000","last_peered":"2024-09-16T14:10:45.616832+0000","last_clean":"2024-09-16T14:10:45.616832+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:10:45.616832+0000","last_undegraded":"2024-09-16T14:10:45.616832+0000","last_fullsized":"2024-09-16T14:10:45.616832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001012095,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4560,"num_read_kb":16131,"num_write":597,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593236+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:10:46.593236+0000","last_peered":"2024-09-16T14:10:46.593236+0000","last_clean":"2024-09-16T14:10:46.593236+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:10:46.593236+0000","last_undegraded":"2024-09-16T14:10:46.593236+0000","last_fullsized":"2024-09-16T14:10:46.593236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"197'780","reported_seq":9812,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573395+0000","last_change":"2024-09-16T14:10:29.447893+0000","last_active":"2024-09-16T14:10:44.573395+0000","last_peered":"2024-09-16T14:10:44.573395+0000","last_clean":"2024-09-16T14:10:44.573395+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:10:44.573395+0000","last_undegraded":"2024-09-16T14:10:44.573395+0000","last_fullsized":"2024-09-16T14:10:44.573395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064281,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":11041,"num_read_kb":12619,"num_write":857,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"198'1412","reported_seq":5233,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590076+0000","last_change":"2024-09-16T14:10:29.449673+0000","last_active":"2024-09-16T14:10:45.590076+0000","last_peered":"2024-09-16T14:10:45.590076+0000","last_clean":"2024-09-16T14:10:45.590076+0000","last_became_active":"2024-09-16T14:10:19.505692+0000","last_became_peered":"2024-09-16T14:10:19.505692+0000","last_unstale":"2024-09-16T14:10:45.590076+0000","last_undegraded":"2024-09-16T14:10:45.590076+0000","last_fullsized":"2024-09-16T14:10:45.590076+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:54.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040141910000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":9687,"num_read_kb":19616,"num_write":1641,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617596+0000","last_change":"2024-09-16T14:10:00.110539+0000","last_active":"2024-09-16T14:10:45.617596+0000","last_peered":"2024-09-16T14:10:45.617596+0000","last_clean":"2024-09-16T14:10:45.617596+0000","last_became_active":"2024-09-16T14:10:00.104539+0000","last_became_peered":"2024-09-16T14:10:00.104539+0000","last_unstale":"2024-09-16T14:10:45.617596+0000","last_undegraded":"2024-09-16T14:10:45.617596+0000","last_fullsized":"2024-09-16T14:10:45.617596+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:07.278197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"196'1316","reported_seq":6546,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592682+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592682+0000","last_peered":"2024-09-16T14:10:46.592682+0000","last_clean":"2024-09-16T14:10:46.592682+0000","last_became_active":"2024-09-16T14:10:26.385527+0000","last_became_peered":"2024-09-16T14:10:26.385527+0000","last_unstale":"2024-09-16T14:10:46.592682+0000","last_undegraded":"2024-09-16T14:10:46.592682+0000","last_fullsized":"2024-09-16T14:10:46.592682+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1316,"log_dups_size":0,"ondisk_log_size":1316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:03.910845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051618139999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9488,"num_read_kb":24272,"num_write":1738,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":8428888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617802+0000","last_change":"2024-09-16T14:10:21.388025+0000","last_active":"2024-09-16T14:10:45.617802+0000","last_peered":"2024-09-16T14:10:45.617802+0000","last_clean":"2024-09-16T14:10:45.617802+0000","last_became_active":"2024-09-16T14:10:21.387173+0000","last_became_peered":"2024-09-16T14:10:21.387173+0000","last_unstale":"2024-09-16T14:10:45.617802+0000","last_undegraded":"2024-09-16T14:10:45.617802+0000","last_fullsized":"2024-09-16T14:10:45.617802+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:57.862024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"197'820","reported_seq":7030,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616505+0000","last_change":"2024-09-16T14:10:29.574667+0000","last_active":"2024-09-16T14:10:45.616505+0000","last_peered":"2024-09-16T14:10:45.616505+0000","last_clean":"2024-09-16T14:10:45.616505+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:10:45.616505+0000","last_undegraded":"2024-09-16T14:10:45.616505+0000","last_fullsized":"2024-09-16T14:10:45.616505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233734,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":8272,"num_read_kb":21183,"num_write":947,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617104+0000","last_change":"2024-09-16T14:10:00.104224+0000","last_active":"2024-09-16T14:10:45.617104+0000","last_peered":"2024-09-16T14:10:45.617104+0000","last_clean":"2024-09-16T14:10:45.617104+0000","last_became_active":"2024-09-16T14:10:00.104003+0000","last_became_peered":"2024-09-16T14:10:00.104003+0000","last_unstale":"2024-09-16T14:10:45.617104+0000","last_undegraded":"2024-09-16T14:10:45.617104+0000","last_fullsized":"2024-09-16T14:10:45.617104+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:08.523062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"197'667","reported_seq":15108,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589347+0000","last_change":"2024-09-16T14:10:29.448082+0000","last_active":"2024-09-16T14:10:45.589347+0000","last_peered":"2024-09-16T14:10:45.589347+0000","last_clean":"2024-09-16T14:10:45.589347+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:10:45.589347+0000","last_undegraded":"2024-09-16T14:10:45.589347+0000","last_fullsized":"2024-09-16T14:10:45.589347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:15.668078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020715640000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":14807,"num_read_kb":52699,"num_write":621,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":56,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617295+0000","last_change":"2024-09-16T14:10:00.112462+0000","last_active":"2024-09-16T14:10:45.617295+0000","last_peered":"2024-09-16T14:10:45.617295+0000","last_clean":"2024-09-16T14:10:45.617295+0000","last_became_active":"2024-09-16T14:10:00.105302+0000","last_became_peered":"2024-09-16T14:10:00.105302+0000","last_unstale":"2024-09-16T14:10:45.617295+0000","last_undegraded":"2024-09-16T14:10:45.617295+0000","last_fullsized":"2024-09-16T14:10:45.617295+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:34:48.094821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591766+0000","last_change":"2024-09-16T14:09:14.399362+0000","last_active":"2024-09-16T14:10:46.591766+0000","last_peered":"2024-09-16T14:10:46.591766+0000","last_clean":"2024-09-16T14:10:46.591766+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:10:46.591766+0000","last_undegraded":"2024-09-16T14:10:46.591766+0000","last_fullsized":"2024-09-16T14:10:46.591766+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:18.538987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"198'704","reported_seq":4286,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589250+0000","last_change":"2024-09-16T14:10:29.446791+0000","last_active":"2024-09-16T14:10:45.589250+0000","last_peered":"2024-09-16T14:10:45.589250+0000","last_clean":"2024-09-16T14:10:45.589250+0000","last_became_active":"2024-09-16T14:10:26.384950+0000","last_became_peered":"2024-09-16T14:10:26.384950+0000","last_unstale":"2024-09-16T14:10:45.589250+0000","last_undegraded":"2024-09-16T14:10:45.589250+0000","last_fullsized":"2024-09-16T14:10:45.589250+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:46.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088242400000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4245,"num_read_kb":11739,"num_write":722,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":2087145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"198'623","reported_seq":6962,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617420+0000","last_change":"2024-09-16T14:10:29.578958+0000","last_active":"2024-09-16T14:10:45.617420+0000","last_peered":"2024-09-16T14:10:45.617420+0000","last_clean":"2024-09-16T14:10:45.617420+0000","last_became_active":"2024-09-16T14:10:08.396086+0000","last_became_peered":"2024-09-16T14:10:08.396086+0000","last_unstale":"2024-09-16T14:10:45.617420+0000","last_undegraded":"2024-09-16T14:10:45.617420+0000","last_fullsized":"2024-09-16T14:10:45.617420+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:00:20.827364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00321149,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6657,"num_read_kb":16604,"num_write":570,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":10317335,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"197'850","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617685+0000","last_change":"2024-09-16T14:10:29.580121+0000","last_active":"2024-09-16T14:10:45.617685+0000","last_peered":"2024-09-16T14:10:45.617685+0000","last_clean":"2024-09-16T14:10:45.617685+0000","last_became_active":"2024-09-16T14:10:07.390964+0000","last_became_peered":"2024-09-16T14:10:07.390964+0000","last_unstale":"2024-09-16T14:10:45.617685+0000","last_undegraded":"2024-09-16T14:10:45.617685+0000","last_fullsized":"2024-09-16T14:10:45.617685+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:48.546737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041712620000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":7231,"num_read_kb":39110,"num_write":775,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":49748621,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"196'747","reported_seq":5182,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617930+0000","last_change":"2024-09-16T14:10:29.457917+0000","last_active":"2024-09-16T14:10:45.617930+0000","last_peered":"2024-09-16T14:10:45.617930+0000","last_clean":"2024-09-16T14:10:45.617930+0000","last_became_active":"2024-09-16T14:10:24.410718+0000","last_became_peered":"2024-09-16T14:10:24.410718+0000","last_unstale":"2024-09-16T14:10:45.617930+0000","last_undegraded":"2024-09-16T14:10:45.617930+0000","last_fullsized":"2024-09-16T14:10:45.617930+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:31.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064224900000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7094,"num_read_kb":44664,"num_write":801,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":19444502,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"197'621","reported_seq":8134,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590037+0000","last_change":"2024-09-16T14:10:29.448787+0000","last_active":"2024-09-16T14:10:45.590037+0000","last_peered":"2024-09-16T14:10:45.590037+0000","last_clean":"2024-09-16T14:10:45.590037+0000","last_became_active":"2024-09-16T14:10:25.384634+0000","last_became_peered":"2024-09-16T14:10:25.384634+0000","last_unstale":"2024-09-16T14:10:45.590037+0000","last_undegraded":"2024-09-16T14:10:45.590037+0000","last_fullsized":"2024-09-16T14:10:45.590037+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:34.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034031370000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7377,"num_read_kb":27693,"num_write":494,"num_write_kb":13033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":23836787,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"197'496","reported_seq":2794,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617334+0000","last_change":"2024-09-16T14:10:29.581839+0000","last_active":"2024-09-16T14:10:45.617334+0000","last_peered":"2024-09-16T14:10:45.617334+0000","last_clean":"2024-09-16T14:10:45.617334+0000","last_became_active":"2024-09-16T14:10:04.123286+0000","last_became_peered":"2024-09-16T14:10:04.123286+0000","last_unstale":"2024-09-16T14:10:45.617334+0000","last_undegraded":"2024-09-16T14:10:45.617334+0000","last_fullsized":"2024-09-16T14:10:45.617334+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:39.608895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054803270000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":25169,"num_write":396,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":20371845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"198'510","reported_seq":7398,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-16T14:10:44.573799+0000","last_change":"2024-09-16T14:10:29.446896+0000","last_active":"2024-09-16T14:10:44.573799+0000","last_peered":"2024-09-16T14:10:44.573799+0000","last_clean":"2024-09-16T14:10:44.573799+0000","last_became_active":"2024-09-16T14:10:16.487713+0000","last_became_peered":"2024-09-16T14:10:16.487713+0000","last_unstale":"2024-09-16T14:10:44.573799+0000","last_undegraded":"2024-09-16T14:10:44.573799+0000","last_fullsized":"2024-09-16T14:10:44.573799+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:51.235912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072767700000000003,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7823,"num_read_kb":31641,"num_write":482,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"197'1003","reported_seq":6899,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590342+0000","last_change":"2024-09-16T14:10:29.450285+0000","last_active":"2024-09-16T14:10:45.590342+0000","last_peered":"2024-09-16T14:10:45.590342+0000","last_clean":"2024-09-16T14:10:45.590342+0000","last_became_active":"2024-09-16T14:10:19.501036+0000","last_became_peered":"2024-09-16T14:10:19.501036+0000","last_unstale":"2024-09-16T14:10:45.590342+0000","last_undegraded":"2024-09-16T14:10:45.590342+0000","last_fullsized":"2024-09-16T14:10:45.590342+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:10.222500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041967539999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10106,"num_read_kb":24848,"num_write":1204,"num_write_kb":15655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":8219627,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"193'625","reported_seq":12640,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591811+0000","last_change":"2024-09-16T14:10:29.462983+0000","last_active":"2024-09-16T14:10:46.591811+0000","last_peered":"2024-09-16T14:10:46.591811+0000","last_clean":"2024-09-16T14:10:46.591811+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:10:46.591811+0000","last_undegraded":"2024-09-16T14:10:46.591811+0000","last_fullsized":"2024-09-16T14:10:46.591811+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:30.500273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114802,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13307,"num_read_kb":24046,"num_write":748,"num_write_kb":16341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"196'986","reported_seq":6243,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617242+0000","last_change":"2024-09-16T14:10:29.578998+0000","last_active":"2024-09-16T14:10:45.617242+0000","last_peered":"2024-09-16T14:10:45.617242+0000","last_clean":"2024-09-16T14:10:45.617242+0000","last_became_active":"2024-09-16T14:10:08.399369+0000","last_became_peered":"2024-09-16T14:10:08.399369+0000","last_unstale":"2024-09-16T14:10:45.617242+0000","last_undegraded":"2024-09-16T14:10:45.617242+0000","last_fullsized":"2024-09-16T14:10:45.617242+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:14.789387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036375169999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6839,"num_read_kb":16656,"num_write":1300,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":7816767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"198'741","reported_seq":6825,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616511+0000","last_change":"2024-09-16T14:10:29.576917+0000","last_active":"2024-09-16T14:10:45.616511+0000","last_peered":"2024-09-16T14:10:45.616511+0000","last_clean":"2024-09-16T14:10:45.616511+0000","last_became_active":"2024-09-16T14:10:05.121914+0000","last_became_peered":"2024-09-16T14:10:05.121914+0000","last_unstale":"2024-09-16T14:10:45.616511+0000","last_undegraded":"2024-09-16T14:10:45.616511+0000","last_fullsized":"2024-09-16T14:10:45.616511+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:38.113333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025226160000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7908,"num_read_kb":56867,"num_write":705,"num_write_kb":31085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":28924045,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"193'753","reported_seq":7845,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617139+0000","last_change":"2024-09-16T14:10:29.580755+0000","last_active":"2024-09-16T14:10:45.617139+0000","last_peered":"2024-09-16T14:10:45.617139+0000","last_clean":"2024-09-16T14:10:45.617139+0000","last_became_active":"2024-09-16T14:10:21.385616+0000","last_became_peered":"2024-09-16T14:10:21.385616+0000","last_unstale":"2024-09-16T14:10:45.617139+0000","last_undegraded":"2024-09-16T14:10:45.617139+0000","last_fullsized":"2024-09-16T14:10:45.617139+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:51.854260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047871169999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8482,"num_read_kb":31842,"num_write":873,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":41601355,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":588,"num_read":281369,"num_read_kb":918178,"num_write":31085,"num_write_kb":441735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9458,"num_bytes_recovered":336276108,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37982,"internal_metadata":0},"log_size":27091,"ondisk_log_size":27091,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304052,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067788,"statfs":{"total":96636764160,"available":96325414912,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.92100000000000004,"15min":0.90900000000000003},"min":{"1min":0.54700000000000004,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.4790000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.91100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.25,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.621}]},{"osd":1,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85199999999999998,"15min":0.85999999999999999},"min":{"1min":0.41199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0409999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.341,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1439999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.52900000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2529999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.371,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.76800000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0149999999999999,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2689999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.375},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.0089999999999999},"min":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.252}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.5549999999999999,"5min":3.246,"15min":3.246},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.024,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2909999999999999,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.52}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.0249999999999999},"min":{"1min":0.63700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.165,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.042},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0189999999999999},"min":{"1min":0.47199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5229999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":1.5840000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0840000000000001,"15min":1.069},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4650000000000001,"5min":3.492,"15min":3.492},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.123,"15min":1.131},"min":{"1min":0.59899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.577,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.6819999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312352,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059488,"statfs":{"total":96636764160,"available":96316915712,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1850000000000001,"5min":1.498,"15min":1.498},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.98799999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2729999999999999,"5min":1.413,"15min":1.413},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95499999999999996,"15min":0.995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.77400000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84099999999999997,"15min":0.85699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.872},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.175,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.014}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.577,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88400000000000001,"15min":0.95399999999999996},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1200000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94199999999999995,"15min":0.97199999999999998},"min":{"1min":0.48299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.56100000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.976,"5min":4.976,"15min":4.976},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.004,"5min":1.03,"15min":1.0549999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.95599999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.55000000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.4319999999999999,"5min":3.4319999999999999,"15min":3.4319999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.101,"15min":1.1419999999999999},"min":{"1min":0.57399999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.089}]}]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304116,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067724,"statfs":{"total":96636764160,"available":96325349376,"internally_reserved":0,"allocated":2654208,"data_stored":1841193,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97299999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.256},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.78700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.079}]},{"osd":1,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.85399999999999998,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.048,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90300000000000002,"15min":0.95499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0900000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.153}]},{"osd":2,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.94199999999999995,"15min":0.92300000000000004},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1479999999999999,"5min":10.999000000000001,"15min":10.999000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":11.018000000000001,"15min":11.018000000000001},"last":1.4159999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.212,"15min":1.212},"last":1.3089999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.93799999999999994},"min":{"1min":0.60399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.601,"15min":1.601},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.96699999999999997,"15min":0.996},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.244,"5min":1.764,"15min":1.764},"last":1.2070000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.538,"15min":1.538},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.999,"15min":1.0580000000000001},"min":{"1min":0.67200000000000004,"5min":0.315,"15min":0.315},"max":{"1min":1.409,"5min":1.573,"15min":1.573},"last":1.1339999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.008,"15min":1.046},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3100000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.0660000000000001,"15min":1.1220000000000001},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3759999999999999,"5min":1.619,"15min":1.619},"last":1.46}]}]},{"osd":3,"up_from":83,"seq":356482285598,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309916,"kb_used_data":2512,"kb_used_omap":31,"kb_used_meta":307360,"kb_avail":94061924,"statfs":{"total":96636764160,"available":96319410176,"internally_reserved":0,"allocated":2572288,"data_stored":1239223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31864,"internal_metadata":314737544},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.36699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":1.468,"15min":1.468},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1459999999999999,"5min":1.258,"15min":1.258},"last":0.64600000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93200000000000005,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.89600000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.119,"5min":1.347,"15min":1.347},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.96399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.2250000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95999999999999996,"15min":0.95299999999999996},"min":{"1min":0.64900000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.1439999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0349999999999999,"15min":1.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.03},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.042},"min":{"1min":0.72099999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.454,"5min":1.486,"15min":1.486},"last":1.202}]},{"osd":7,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.024,"15min":1.024},"min":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2929999999999999,"5min":1.419,"15min":1.419},"last":1.306},{"interface":"front","average":{"1min":1.103,"5min":1.0369999999999999,"15min":1.026},"min":{"1min":0.77500000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4139999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.2729999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308184,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063656,"statfs":{"total":96636764160,"available":96321183744,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.276,"5min":1.454,"15min":1.454},"last":0.72099999999999997}]},{"osd":3,"last update":"Mon Sep 16 14:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.5,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.753}]},{"osd":4,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.878}]},{"osd":5,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4390000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.139},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.948,"15min":1.948},"last":1.1839999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92700000000000005,"15min":0.93200000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.026,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.415,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.008}]},{"osd":7,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.98799999999999999,"15min":0.999},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.623,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.022,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4470000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.0469999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312424,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059416,"statfs":{"total":96636764160,"available":96316841984,"internally_reserved":0,"allocated":2637824,"data_stored":1841193,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.80800000000000005,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.5890000000000004,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.89800000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.70299999999999996,"15min":0.745},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.90500000000000003,"5min":1.998,"15min":1.998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Mon Sep 16 14:10:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":7.9950000000000001,"5min":7.9950000000000001,"15min":7.9950000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0009999999999999,"5min":1.244,"15min":1.244},"last":0.82199999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.89200000000000002},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.89600000000000002,"15min":0.92600000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1359999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.087,"15min":1.1200000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2330000000000001,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":1.034},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.077,"15min":1.097},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0600000000000001,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0429999999999999,"15min":1.0960000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.874}]},{"osd":7,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.018,"15min":1.0649999999999999},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.865,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.125},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.145,"15min":1.149},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.375,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":1.1930000000000001}]}]},{"osd":0,"up_from":184,"seq":790273982473,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311836,"kb_used_data":2508,"kb_used_omap":125,"kb_used_meta":309186,"kb_avail":94060004,"statfs":{"total":96636764160,"available":96317444096,"internally_reserved":0,"allocated":2568192,"data_stored":1239204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":128171,"internal_metadata":316607317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542196,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316520,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94055320,"statfs":{"total":96636764160,"available":96312647680,"internally_reserved":0,"allocated":2580480,"data_stored":1245005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2809999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.268},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1990000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.2210000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.121,"5min":37.805,"15min":37.805},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1819999999999999,"15min":1.008},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.155,"5min":37.857999999999997,"15min":37.857999999999997},"last":0.91300000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.115,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.204,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1100000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:10:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.68000000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76200000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.133,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.60999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2829999999999999,"5min":1.611,"15min":1.611},"last":0.83499999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.95899999999999996,"15min":0.97299999999999998},"min":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3009999999999999,"5min":1.429,"15min":1.429},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.67700000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.278,"5min":1.601,"15min":1.601},"last":1.383}]},{"osd":7,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68700000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.355,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0780000000000001,"15min":1.097},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3819999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.236}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29034,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:10:52.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:10:52.948 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:52.948 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":215,"fsid":"a22b38a6-94db-470d-bf35-45d7605f3bb0","created":"2024-09-16T14:06:40.573726+0000","modified":"2024-09-16T14:10:47.595917+0000","last_up_change":"2024-09-16T14:10:15.439434+0000","last_in_change":"2024-09-16T14:10:20.368208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:06:52.822826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:06:58.526935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T14:07:08.375919+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":103,"snap_epoch":198,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a060b161-fd5a-4cfb-bac8-f2acc32d7955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":129,"up_from":184,"up_thru":193,"down_at":130,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":47342},{"type":"v1","addr":"172.21.15.79:6817","nonce":47342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":47342},{"type":"v1","addr":"172.21.15.79:6819","nonce":47342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":47342},{"type":"v1","addr":"172.21.15.79:6824","nonce":47342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":47342},{"type":"v1","addr":"172.21.15.79:6821","nonce":47342}]},"public_addr":"172.21.15.79:6817/47342","cluster_addr":"172.21.15.79:6819/47342","heartbeat_back_addr":"172.21.15.79:6824/47342","heartbeat_front_addr":"172.21.15.79:6821/47342","state":["exists","up"]},{"osd":1,"uuid":"f5ba4b39-ceb8-46b3-938e-57249ed84b87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":188,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":22195},{"type":"v1","addr":"172.21.15.79:6809","nonce":22195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":22195},{"type":"v1","addr":"172.21.15.79:6811","nonce":22195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":22195},{"type":"v1","addr":"172.21.15.79:6815","nonce":22195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":22195},{"type":"v1","addr":"172.21.15.79:6813","nonce":22195}]},"public_addr":"172.21.15.79:6809/22195","cluster_addr":"172.21.15.79:6811/22195","heartbeat_back_addr":"172.21.15.79:6815/22195","heartbeat_front_addr":"172.21.15.79:6813/22195","state":["exists","up"]},{"osd":2,"uuid":"8b657f92-66c9-46a3-a31f-0bd0be414e4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6823","nonce":22197},{"type":"v1","addr":"172.21.15.79:6825","nonce":22197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":22197},{"type":"v1","addr":"172.21.15.79:6827","nonce":22197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":22197},{"type":"v1","addr":"172.21.15.79:6831","nonce":22197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":22197},{"type":"v1","addr":"172.21.15.79:6829","nonce":22197}]},"public_addr":"172.21.15.79:6825/22197","cluster_addr":"172.21.15.79:6827/22197","heartbeat_back_addr":"172.21.15.79:6831/22197","heartbeat_front_addr":"172.21.15.79:6829/22197","state":["exists","up"]},{"osd":3,"uuid":"15937bd7-8964-4140-9f86-b3093c318d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":83,"up_thru":189,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":35927},{"type":"v1","addr":"172.21.15.79:6801","nonce":35927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":35927},{"type":"v1","addr":"172.21.15.79:6803","nonce":35927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":35927},{"type":"v1","addr":"172.21.15.79:6807","nonce":35927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":35927},{"type":"v1","addr":"172.21.15.79:6805","nonce":35927}]},"public_addr":"172.21.15.79:6801/35927","cluster_addr":"172.21.15.79:6803/35927","heartbeat_back_addr":"172.21.15.79:6807/35927","heartbeat_front_addr":"172.21.15.79:6805/35927","state":["exists","up"]},{"osd":4,"uuid":"714baf80-ade0-4e88-ad4e-5171add1fc6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":22362},{"type":"v1","addr":"172.21.15.164:6809","nonce":22362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":22362},{"type":"v1","addr":"172.21.15.164:6811","nonce":22362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":22362},{"type":"v1","addr":"172.21.15.164:6815","nonce":22362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":22362},{"type":"v1","addr":"172.21.15.164:6813","nonce":22362}]},"public_addr":"172.21.15.164:6809/22362","cluster_addr":"172.21.15.164:6811/22362","heartbeat_back_addr":"172.21.15.164:6815/22362","heartbeat_front_addr":"172.21.15.164:6813/22362","state":["exists","up"]},{"osd":5,"uuid":"e5a21b84-81ab-45da-a081-3b504f5c4f3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":22363},{"type":"v1","addr":"172.21.15.164:6801","nonce":22363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":22363},{"type":"v1","addr":"172.21.15.164:6803","nonce":22363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":22363},{"type":"v1","addr":"172.21.15.164:6807","nonce":22363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":22363},{"type":"v1","addr":"172.21.15.164:6805","nonce":22363}]},"public_addr":"172.21.15.164:6801/22363","cluster_addr":"172.21.15.164:6803/22363","heartbeat_back_addr":"172.21.15.164:6807/22363","heartbeat_front_addr":"172.21.15.164:6805/22363","state":["exists","up"]},{"osd":6,"uuid":"a009857f-f798-4154-9ff4-f524f60b6c61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":22364},{"type":"v1","addr":"172.21.15.164:6825","nonce":22364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":22364},{"type":"v1","addr":"172.21.15.164:6827","nonce":22364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":22364},{"type":"v1","addr":"172.21.15.164:6831","nonce":22364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":22364},{"type":"v1","addr":"172.21.15.164:6829","nonce":22364}]},"public_addr":"172.21.15.164:6825/22364","cluster_addr":"172.21.15.164:6827/22364","heartbeat_back_addr":"172.21.15.164:6831/22364","heartbeat_front_addr":"172.21.15.164:6829/22364","state":["exists","up"]},{"osd":7,"uuid":"e3506002-ba8b-4251-8ef1-136334574a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":22365},{"type":"v1","addr":"172.21.15.164:6817","nonce":22365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":22365},{"type":"v1","addr":"172.21.15.164:6819","nonce":22365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":22365},{"type":"v1","addr":"172.21.15.164:6823","nonce":22365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":22365},{"type":"v1","addr":"172.21.15.164:6821","nonce":22365}]},"public_addr":"172.21.15.164:6817/22365","cluster_addr":"172.21.15.164:6819/22365","heartbeat_back_addr":"172.21.15.164:6823/22365","heartbeat_front_addr":"172.21.15.164:6821/22365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.104631+0000","dead_epoch":130},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987403+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.018134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.987645+0000","dead_epoch":59},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.721507+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:47.869796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.206576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:06:48.038098+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:10:53.964 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T14:10:53.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:10:54.126 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:54.126 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:10:54.126 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:54.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T14:10:54.440 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T14:10:54.450 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T14:10:54.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:10:54.596 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:54.596 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:10:54.596 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:54.609 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T14:10:54.900 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T14:10:54.910 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T14:10:54.910 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:10:55.053 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:55.053 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:10:55.053 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:55.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T14:10:55.362 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T14:10:55.372 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T14:10:55.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:10:55.515 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:55.515 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:10:55.516 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:55.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T14:10:55.825 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T14:10:55.838 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T14:10:55.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:10:55.975 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:10:55.975 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:55.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T14:10:56.292 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T14:10:56.302 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T14:10:56.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:10:56.445 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:56.445 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:10:56.445 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:56.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T14:10:56.749 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T14:10:56.759 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T14:10:56.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:10:56.903 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:56.903 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:10:56.903 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:56.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T14:10:57.245 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T14:10:57.258 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T14:10:57.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:10:57.399 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-09-16T14:10:57.400 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:10:57.400 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-09-16T14:10:57.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T14:10:57.704 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T14:10:57.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:10:57.997 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:10:57.999 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:10:58.012 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-16T14:10:56.832056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":588,"num_read":281435,"num_read_kb":918233,"num_write":31188,"num_write_kb":443365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9465,"num_bytes_recovered":336866533,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27166,"ondisk_log_size":27166,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2479500,"kb_used_data":20368,"kb_used_omap":166,"kb_used_meta":2458841,"kb_avail":752495220,"statfs":{"total":773094113280,"available":770555105280,"internally_reserved":0,"allocated":20856832,"data_stored":11187260,"data_compressed":11598,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":170259,"internal_metadata":2517853933},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002616"},"pg_stats":[{"pgid":"3.1f","version":"198'1876","reported_seq":7887,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592881+0000","last_change":"2024-09-16T14:10:29.531167+0000","last_active":"2024-09-16T14:10:46.592881+0000","last_peered":"2024-09-16T14:10:46.592881+0000","last_clean":"2024-09-16T14:10:46.592881+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:10:46.592881+0000","last_undegraded":"2024-09-16T14:10:46.592881+0000","last_fullsized":"2024-09-16T14:10:46.592881+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1876,"log_dups_size":0,"ondisk_log_size":1876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:02.959438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039454019999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15599,"num_read_kb":19767,"num_write":2973,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"196'800","reported_seq":2726,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592990+0000","last_change":"2024-09-16T14:10:29.528090+0000","last_active":"2024-09-16T14:10:46.592990+0000","last_peered":"2024-09-16T14:10:46.592990+0000","last_clean":"2024-09-16T14:10:46.592990+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:10:46.592990+0000","last_undegraded":"2024-09-16T14:10:46.592990+0000","last_fullsized":"2024-09-16T14:10:46.592990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:15.543030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098036900000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3428,"num_read_kb":21398,"num_write":967,"num_write_kb":11613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"197'653","reported_seq":5531,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865843+0000","last_change":"2024-09-16T14:10:29.581527+0000","last_active":"2024-09-16T14:10:47.865843+0000","last_peered":"2024-09-16T14:10:47.865843+0000","last_clean":"2024-09-16T14:10:47.865843+0000","last_became_active":"2024-09-16T14:10:20.387629+0000","last_became_peered":"2024-09-16T14:10:20.387629+0000","last_unstale":"2024-09-16T14:10:47.865843+0000","last_undegraded":"2024-09-16T14:10:47.865843+0000","last_fullsized":"2024-09-16T14:10:47.865843+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051863780000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7537,"num_read_kb":24768,"num_write":780,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":26063912,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"198'1032","reported_seq":14527,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592725+0000","last_change":"2024-09-16T14:10:29.463373+0000","last_active":"2024-09-16T14:10:46.592725+0000","last_peered":"2024-09-16T14:10:46.592725+0000","last_clean":"2024-09-16T14:10:46.592725+0000","last_became_active":"2024-09-16T14:10:20.385216+0000","last_became_peered":"2024-09-16T14:10:20.385216+0000","last_unstale":"2024-09-16T14:10:46.592725+0000","last_undegraded":"2024-09-16T14:10:46.592725+0000","last_fullsized":"2024-09-16T14:10:46.592725+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1032,"log_dups_size":0,"ondisk_log_size":1032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:22.705027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013844,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":21,"num_read":15843,"num_read_kb":26936,"num_write":1369,"num_write_kb":11030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":6143753,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"197'738","reported_seq":8081,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591631+0000","last_change":"2024-09-16T14:10:29.446977+0000","last_active":"2024-09-16T14:10:46.591631+0000","last_peered":"2024-09-16T14:10:46.591631+0000","last_clean":"2024-09-16T14:10:46.591631+0000","last_became_active":"2024-09-16T14:10:20.387007+0000","last_became_peered":"2024-09-16T14:10:20.387007+0000","last_unstale":"2024-09-16T14:10:46.591631+0000","last_undegraded":"2024-09-16T14:10:46.591631+0000","last_fullsized":"2024-09-16T14:10:46.591631+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:41.660168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010460949999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10352,"num_read_kb":26550,"num_write":833,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"196'845","reported_seq":8027,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592011+0000","last_change":"2024-09-16T14:10:29.463712+0000","last_active":"2024-09-16T14:10:46.592011+0000","last_peered":"2024-09-16T14:10:46.592011+0000","last_clean":"2024-09-16T14:10:46.592011+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:10:46.592011+0000","last_undegraded":"2024-09-16T14:10:46.592011+0000","last_fullsized":"2024-09-16T14:10:46.592011+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:25.215853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017171580000000001,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":10449,"num_read_kb":28504,"num_write":803,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"197'630","reported_seq":2343,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591592+0000","last_change":"2024-09-16T14:10:29.447450+0000","last_active":"2024-09-16T14:10:46.591592+0000","last_peered":"2024-09-16T14:10:46.591592+0000","last_clean":"2024-09-16T14:10:46.591592+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:10:46.591592+0000","last_undegraded":"2024-09-16T14:10:46.591592+0000","last_fullsized":"2024-09-16T14:10:46.591592+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:41.369426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483112,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1924,"num_read_kb":28666,"num_write":709,"num_write_kb":21954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"198'968","reported_seq":6364,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592751+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592751+0000","last_peered":"2024-09-16T14:10:46.592751+0000","last_clean":"2024-09-16T14:10:46.592751+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:10:46.592751+0000","last_undegraded":"2024-09-16T14:10:46.592751+0000","last_fullsized":"2024-09-16T14:10:46.592751+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":968,"log_dups_size":0,"ondisk_log_size":968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:13.126243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050019210000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9238,"num_read_kb":21812,"num_write":1162,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"194'1351","reported_seq":6429,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593259+0000","last_change":"2024-09-16T14:10:29.531182+0000","last_active":"2024-09-16T14:10:46.593259+0000","last_peered":"2024-09-16T14:10:46.593259+0000","last_clean":"2024-09-16T14:10:46.593259+0000","last_became_active":"2024-09-16T14:10:05.123005+0000","last_became_peered":"2024-09-16T14:10:05.123005+0000","last_unstale":"2024-09-16T14:10:46.593259+0000","last_undegraded":"2024-09-16T14:10:46.593259+0000","last_fullsized":"2024-09-16T14:10:46.593259+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:44.294281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039366710000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8929,"num_read_kb":33272,"num_write":1987,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":4502377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"197'751","reported_seq":13942,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590143+0000","last_change":"2024-09-16T14:10:29.446978+0000","last_active":"2024-09-16T14:10:45.590143+0000","last_peered":"2024-09-16T14:10:45.590143+0000","last_clean":"2024-09-16T14:10:45.590143+0000","last_became_active":"2024-09-16T14:10:05.377635+0000","last_became_peered":"2024-09-16T14:10:05.377635+0000","last_unstale":"2024-09-16T14:10:45.590143+0000","last_undegraded":"2024-09-16T14:10:45.590143+0000","last_fullsized":"2024-09-16T14:10:45.590143+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:35:42.336862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089383699999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":14525,"num_read_kb":50269,"num_write":689,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":11596768,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"198'576","reported_seq":5190,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:48.679175+0000","last_change":"2024-09-16T14:10:29.448044+0000","last_active":"2024-09-16T14:10:48.679175+0000","last_peered":"2024-09-16T14:10:48.679175+0000","last_clean":"2024-09-16T14:10:48.679175+0000","last_became_active":"2024-09-16T14:10:19.498121+0000","last_became_peered":"2024-09-16T14:10:19.498121+0000","last_unstale":"2024-09-16T14:10:48.679175+0000","last_undegraded":"2024-09-16T14:10:48.679175+0000","last_fullsized":"2024-09-16T14:10:48.679175+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002716544,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4971,"num_read_kb":28564,"num_write":500,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":50024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"197'723","reported_seq":7825,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865561+0000","last_change":"2024-09-16T14:10:29.582367+0000","last_active":"2024-09-16T14:10:47.865561+0000","last_peered":"2024-09-16T14:10:47.865561+0000","last_clean":"2024-09-16T14:10:47.865561+0000","last_became_active":"2024-09-16T14:10:06.388650+0000","last_became_peered":"2024-09-16T14:10:06.388650+0000","last_unstale":"2024-09-16T14:10:47.865561+0000","last_undegraded":"2024-09-16T14:10:47.865561+0000","last_fullsized":"2024-09-16T14:10:47.865561+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054744670000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9236,"num_read_kb":38811,"num_write":682,"num_write_kb":10309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":10174643,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865735+0000","last_change":"2024-09-16T14:10:17.556015+0000","last_active":"2024-09-16T14:10:47.865735+0000","last_peered":"2024-09-16T14:10:47.865735+0000","last_clean":"2024-09-16T14:10:47.865735+0000","last_became_active":"2024-09-16T14:10:17.555698+0000","last_became_peered":"2024-09-16T14:10:17.555698+0000","last_unstale":"2024-09-16T14:10:47.865735+0000","last_undegraded":"2024-09-16T14:10:47.865735+0000","last_fullsized":"2024-09-16T14:10:47.865735+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"198'897","reported_seq":6177,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591871+0000","last_change":"2024-09-16T14:10:29.448811+0000","last_active":"2024-09-16T14:10:46.591871+0000","last_peered":"2024-09-16T14:10:46.591871+0000","last_clean":"2024-09-16T14:10:46.591871+0000","last_became_active":"2024-09-16T14:10:25.385245+0000","last_became_peered":"2024-09-16T14:10:25.385245+0000","last_unstale":"2024-09-16T14:10:46.591871+0000","last_undegraded":"2024-09-16T14:10:46.591871+0000","last_fullsized":"2024-09-16T14:10:46.591871+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:00.028708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026361219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5530,"num_read_kb":25811,"num_write":916,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":731,"num_bytes_recovered":15701912,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591854+0000","last_change":"2024-09-16T14:10:16.472462+0000","last_active":"2024-09-16T14:10:46.591854+0000","last_peered":"2024-09-16T14:10:46.591854+0000","last_clean":"2024-09-16T14:10:46.591854+0000","last_became_active":"2024-09-16T14:10:16.471977+0000","last_became_peered":"2024-09-16T14:10:16.471977+0000","last_unstale":"2024-09-16T14:10:46.591854+0000","last_undegraded":"2024-09-16T14:10:46.591854+0000","last_fullsized":"2024-09-16T14:10:46.591854+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:09.028511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"196'967","reported_seq":7725,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591858+0000","last_change":"2024-09-16T14:10:29.448302+0000","last_active":"2024-09-16T14:10:46.591858+0000","last_peered":"2024-09-16T14:10:46.591858+0000","last_clean":"2024-09-16T14:10:46.591858+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:10:46.591858+0000","last_undegraded":"2024-09-16T14:10:46.591858+0000","last_fullsized":"2024-09-16T14:10:46.591858+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:58.173875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002259535,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":10392,"num_read_kb":45651,"num_write":1244,"num_write_kb":30465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865841+0000","last_change":"2024-09-16T14:10:00.102785+0000","last_active":"2024-09-16T14:10:47.865841+0000","last_peered":"2024-09-16T14:10:47.865841+0000","last_clean":"2024-09-16T14:10:47.865841+0000","last_became_active":"2024-09-16T14:10:00.102521+0000","last_became_peered":"2024-09-16T14:10:00.102521+0000","last_unstale":"2024-09-16T14:10:47.865841+0000","last_undegraded":"2024-09-16T14:10:47.865841+0000","last_fullsized":"2024-09-16T14:10:47.865841+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"197'630","reported_seq":3952,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616832+0000","last_change":"2024-09-16T14:10:29.574272+0000","last_active":"2024-09-16T14:10:45.616832+0000","last_peered":"2024-09-16T14:10:45.616832+0000","last_clean":"2024-09-16T14:10:45.616832+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:10:45.616832+0000","last_undegraded":"2024-09-16T14:10:45.616832+0000","last_fullsized":"2024-09-16T14:10:45.616832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:50:45.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001012095,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4560,"num_read_kb":16131,"num_write":597,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.593236+0000","last_change":"2024-09-16T14:07:02.494285+0000","last_active":"2024-09-16T14:10:46.593236+0000","last_peered":"2024-09-16T14:10:46.593236+0000","last_clean":"2024-09-16T14:10:46.593236+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:10:46.593236+0000","last_undegraded":"2024-09-16T14:10:46.593236+0000","last_fullsized":"2024-09-16T14:10:46.593236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:17.745706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"197'780","reported_seq":9813,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.589954+0000","last_change":"2024-09-16T14:10:29.447893+0000","last_active":"2024-09-16T14:10:45.589954+0000","last_peered":"2024-09-16T14:10:45.589954+0000","last_clean":"2024-09-16T14:10:45.589954+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:10:45.589954+0000","last_undegraded":"2024-09-16T14:10:45.589954+0000","last_fullsized":"2024-09-16T14:10:45.589954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:14.055296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001064281,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":11041,"num_read_kb":12619,"num_write":857,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"198'1412","reported_seq":5234,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:48.679303+0000","last_change":"2024-09-16T14:10:29.449673+0000","last_active":"2024-09-16T14:10:48.679303+0000","last_peered":"2024-09-16T14:10:48.679303+0000","last_clean":"2024-09-16T14:10:48.679303+0000","last_became_active":"2024-09-16T14:10:19.505692+0000","last_became_peered":"2024-09-16T14:10:19.505692+0000","last_unstale":"2024-09-16T14:10:48.679303+0000","last_undegraded":"2024-09-16T14:10:48.679303+0000","last_fullsized":"2024-09-16T14:10:48.679303+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1412,"log_dups_size":0,"ondisk_log_size":1412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040141910000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":9687,"num_read_kb":19616,"num_write":1641,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":57,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865396+0000","last_change":"2024-09-16T14:10:00.110539+0000","last_active":"2024-09-16T14:10:47.865396+0000","last_peered":"2024-09-16T14:10:47.865396+0000","last_clean":"2024-09-16T14:10:47.865396+0000","last_became_active":"2024-09-16T14:10:00.104539+0000","last_became_peered":"2024-09-16T14:10:00.104539+0000","last_unstale":"2024-09-16T14:10:47.865396+0000","last_undegraded":"2024-09-16T14:10:47.865396+0000","last_fullsized":"2024-09-16T14:10:47.865396+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"196'1316","reported_seq":6546,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.592682+0000","last_change":"2024-09-16T14:10:29.532742+0000","last_active":"2024-09-16T14:10:46.592682+0000","last_peered":"2024-09-16T14:10:46.592682+0000","last_clean":"2024-09-16T14:10:46.592682+0000","last_became_active":"2024-09-16T14:10:26.385527+0000","last_became_peered":"2024-09-16T14:10:26.385527+0000","last_unstale":"2024-09-16T14:10:46.592682+0000","last_undegraded":"2024-09-16T14:10:46.592682+0000","last_fullsized":"2024-09-16T14:10:46.592682+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1316,"log_dups_size":0,"ondisk_log_size":1316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:03.910845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051618139999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9488,"num_read_kb":24272,"num_write":1738,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":8428888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617802+0000","last_change":"2024-09-16T14:10:21.388025+0000","last_active":"2024-09-16T14:10:45.617802+0000","last_peered":"2024-09-16T14:10:45.617802+0000","last_clean":"2024-09-16T14:10:45.617802+0000","last_became_active":"2024-09-16T14:10:21.387173+0000","last_became_peered":"2024-09-16T14:10:21.387173+0000","last_unstale":"2024-09-16T14:10:45.617802+0000","last_undegraded":"2024-09-16T14:10:45.617802+0000","last_fullsized":"2024-09-16T14:10:45.617802+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:57.862024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"197'820","reported_seq":7030,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616505+0000","last_change":"2024-09-16T14:10:29.574667+0000","last_active":"2024-09-16T14:10:45.616505+0000","last_peered":"2024-09-16T14:10:45.616505+0000","last_clean":"2024-09-16T14:10:45.616505+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:10:45.616505+0000","last_undegraded":"2024-09-16T14:10:45.616505+0000","last_fullsized":"2024-09-16T14:10:45.616505+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:32.612181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233734,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":8272,"num_read_kb":21183,"num_write":947,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865309+0000","last_change":"2024-09-16T14:10:00.104224+0000","last_active":"2024-09-16T14:10:47.865309+0000","last_peered":"2024-09-16T14:10:47.865309+0000","last_clean":"2024-09-16T14:10:47.865309+0000","last_became_active":"2024-09-16T14:10:00.104003+0000","last_became_peered":"2024-09-16T14:10:00.104003+0000","last_unstale":"2024-09-16T14:10:47.865309+0000","last_undegraded":"2024-09-16T14:10:47.865309+0000","last_fullsized":"2024-09-16T14:10:47.865309+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"197'667","reported_seq":15109,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591593+0000","last_change":"2024-09-16T14:10:29.448082+0000","last_active":"2024-09-16T14:10:46.591593+0000","last_peered":"2024-09-16T14:10:46.591593+0000","last_clean":"2024-09-16T14:10:46.591593+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:10:46.591593+0000","last_undegraded":"2024-09-16T14:10:46.591593+0000","last_fullsized":"2024-09-16T14:10:46.591593+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:15.668078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020715640000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":14807,"num_read_kb":52699,"num_write":621,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":57,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865533+0000","last_change":"2024-09-16T14:10:00.112462+0000","last_active":"2024-09-16T14:10:47.865533+0000","last_peered":"2024-09-16T14:10:47.865533+0000","last_clean":"2024-09-16T14:10:47.865533+0000","last_became_active":"2024-09-16T14:10:00.105302+0000","last_became_peered":"2024-09-16T14:10:00.105302+0000","last_unstale":"2024-09-16T14:10:47.865533+0000","last_undegraded":"2024-09-16T14:10:47.865533+0000","last_fullsized":"2024-09-16T14:10:47.865533+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":103,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591766+0000","last_change":"2024-09-16T14:09:14.399362+0000","last_active":"2024-09-16T14:10:46.591766+0000","last_peered":"2024-09-16T14:10:46.591766+0000","last_clean":"2024-09-16T14:10:46.591766+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:10:46.591766+0000","last_undegraded":"2024-09-16T14:10:46.591766+0000","last_fullsized":"2024-09-16T14:10:46.591766+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:52.823835+0000","last_clean_scrub_stamp":"2024-09-16T14:06:52.823835+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:18.538987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"198'704","reported_seq":4287,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591532+0000","last_change":"2024-09-16T14:10:29.446791+0000","last_active":"2024-09-16T14:10:46.591532+0000","last_peered":"2024-09-16T14:10:46.591532+0000","last_clean":"2024-09-16T14:10:46.591532+0000","last_became_active":"2024-09-16T14:10:26.384950+0000","last_became_peered":"2024-09-16T14:10:26.384950+0000","last_unstale":"2024-09-16T14:10:46.591532+0000","last_undegraded":"2024-09-16T14:10:46.591532+0000","last_fullsized":"2024-09-16T14:10:46.591532+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:46.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088242400000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4245,"num_read_kb":11739,"num_write":722,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":2087145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"198'623","reported_seq":6963,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865095+0000","last_change":"2024-09-16T14:10:29.578958+0000","last_active":"2024-09-16T14:10:47.865095+0000","last_peered":"2024-09-16T14:10:47.865095+0000","last_clean":"2024-09-16T14:10:47.865095+0000","last_became_active":"2024-09-16T14:10:08.396086+0000","last_became_peered":"2024-09-16T14:10:08.396086+0000","last_unstale":"2024-09-16T14:10:47.865095+0000","last_undegraded":"2024-09-16T14:10:47.865095+0000","last_fullsized":"2024-09-16T14:10:47.865095+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00321149,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6657,"num_read_kb":16604,"num_write":570,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":10317335,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"197'850","reported_seq":6900,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865019+0000","last_change":"2024-09-16T14:10:29.580121+0000","last_active":"2024-09-16T14:10:47.865019+0000","last_peered":"2024-09-16T14:10:47.865019+0000","last_clean":"2024-09-16T14:10:47.865019+0000","last_became_active":"2024-09-16T14:10:07.390964+0000","last_became_peered":"2024-09-16T14:10:07.390964+0000","last_unstale":"2024-09-16T14:10:47.865019+0000","last_undegraded":"2024-09-16T14:10:47.865019+0000","last_fullsized":"2024-09-16T14:10:47.865019+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041712620000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":7231,"num_read_kb":39110,"num_write":775,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":49748621,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"196'747","reported_seq":5182,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.617930+0000","last_change":"2024-09-16T14:10:29.457917+0000","last_active":"2024-09-16T14:10:45.617930+0000","last_peered":"2024-09-16T14:10:45.617930+0000","last_clean":"2024-09-16T14:10:45.617930+0000","last_became_active":"2024-09-16T14:10:24.410718+0000","last_became_peered":"2024-09-16T14:10:24.410718+0000","last_unstale":"2024-09-16T14:10:45.617930+0000","last_undegraded":"2024-09-16T14:10:45.617930+0000","last_fullsized":"2024-09-16T14:10:45.617930+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:31.330258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064224900000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7094,"num_read_kb":44664,"num_write":801,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":19444502,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"197'621","reported_seq":8135,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:48.679161+0000","last_change":"2024-09-16T14:10:29.448787+0000","last_active":"2024-09-16T14:10:48.679161+0000","last_peered":"2024-09-16T14:10:48.679161+0000","last_clean":"2024-09-16T14:10:48.679161+0000","last_became_active":"2024-09-16T14:10:25.384634+0000","last_became_peered":"2024-09-16T14:10:25.384634+0000","last_unstale":"2024-09-16T14:10:48.679161+0000","last_undegraded":"2024-09-16T14:10:48.679161+0000","last_fullsized":"2024-09-16T14:10:48.679161+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034031370000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7377,"num_read_kb":27693,"num_write":494,"num_write_kb":13033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":23836787,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"197'496","reported_seq":2795,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865603+0000","last_change":"2024-09-16T14:10:29.581839+0000","last_active":"2024-09-16T14:10:47.865603+0000","last_peered":"2024-09-16T14:10:47.865603+0000","last_clean":"2024-09-16T14:10:47.865603+0000","last_became_active":"2024-09-16T14:10:04.123286+0000","last_became_peered":"2024-09-16T14:10:04.123286+0000","last_unstale":"2024-09-16T14:10:47.865603+0000","last_undegraded":"2024-09-16T14:10:47.865603+0000","last_fullsized":"2024-09-16T14:10:47.865603+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054803270000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":25169,"num_write":396,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":20371845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"198'510","reported_seq":7399,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.590062+0000","last_change":"2024-09-16T14:10:29.446896+0000","last_active":"2024-09-16T14:10:45.590062+0000","last_peered":"2024-09-16T14:10:45.590062+0000","last_clean":"2024-09-16T14:10:45.590062+0000","last_became_active":"2024-09-16T14:10:16.487713+0000","last_became_peered":"2024-09-16T14:10:16.487713+0000","last_unstale":"2024-09-16T14:10:45.590062+0000","last_undegraded":"2024-09-16T14:10:45.590062+0000","last_fullsized":"2024-09-16T14:10:45.590062+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:51.235912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072767700000000003,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7823,"num_read_kb":31641,"num_write":482,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"197'1003","reported_seq":6900,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:48.679390+0000","last_change":"2024-09-16T14:10:29.450285+0000","last_active":"2024-09-16T14:10:48.679390+0000","last_peered":"2024-09-16T14:10:48.679390+0000","last_clean":"2024-09-16T14:10:48.679390+0000","last_became_active":"2024-09-16T14:10:19.501036+0000","last_became_peered":"2024-09-16T14:10:19.501036+0000","last_unstale":"2024-09-16T14:10:48.679390+0000","last_undegraded":"2024-09-16T14:10:48.679390+0000","last_fullsized":"2024-09-16T14:10:48.679390+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041967539999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10106,"num_read_kb":24848,"num_write":1204,"num_write_kb":15655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":8219627,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"193'625","reported_seq":12640,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T14:10:46.591811+0000","last_change":"2024-09-16T14:10:29.462983+0000","last_active":"2024-09-16T14:10:46.591811+0000","last_peered":"2024-09-16T14:10:46.591811+0000","last_clean":"2024-09-16T14:10:46.591811+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:10:46.591811+0000","last_undegraded":"2024-09-16T14:10:46.591811+0000","last_fullsized":"2024-09-16T14:10:46.591811+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:30.500273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114802,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13307,"num_read_kb":24046,"num_write":748,"num_write_kb":16341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"196'986","reported_seq":6244,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865051+0000","last_change":"2024-09-16T14:10:29.578998+0000","last_active":"2024-09-16T14:10:47.865051+0000","last_peered":"2024-09-16T14:10:47.865051+0000","last_clean":"2024-09-16T14:10:47.865051+0000","last_became_active":"2024-09-16T14:10:08.399369+0000","last_became_peered":"2024-09-16T14:10:08.399369+0000","last_unstale":"2024-09-16T14:10:47.865051+0000","last_undegraded":"2024-09-16T14:10:47.865051+0000","last_fullsized":"2024-09-16T14:10:47.865051+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036375169999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6839,"num_read_kb":16656,"num_write":1300,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":7816767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"198'741","reported_seq":6825,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-16T14:10:45.616511+0000","last_change":"2024-09-16T14:10:29.576917+0000","last_active":"2024-09-16T14:10:45.616511+0000","last_peered":"2024-09-16T14:10:45.616511+0000","last_clean":"2024-09-16T14:10:45.616511+0000","last_became_active":"2024-09-16T14:10:05.121914+0000","last_became_peered":"2024-09-16T14:10:05.121914+0000","last_unstale":"2024-09-16T14:10:45.616511+0000","last_undegraded":"2024-09-16T14:10:45.616511+0000","last_fullsized":"2024-09-16T14:10:45.616511+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:38.113333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025226160000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7908,"num_read_kb":56867,"num_write":705,"num_write_kb":31085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":28924045,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"193'753","reported_seq":7846,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865195+0000","last_change":"2024-09-16T14:10:29.580755+0000","last_active":"2024-09-16T14:10:47.865195+0000","last_peered":"2024-09-16T14:10:47.865195+0000","last_clean":"2024-09-16T14:10:47.865195+0000","last_became_active":"2024-09-16T14:10:21.385616+0000","last_became_peered":"2024-09-16T14:10:21.385616+0000","last_unstale":"2024-09-16T14:10:47.865195+0000","last_undegraded":"2024-09-16T14:10:47.865195+0000","last_fullsized":"2024-09-16T14:10:47.865195+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047871169999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8482,"num_read_kb":31842,"num_write":873,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":41601355,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":588,"num_read":281369,"num_read_kb":918178,"num_write":31085,"num_write_kb":441735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9458,"num_bytes_recovered":336276108,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37982,"internal_metadata":0},"log_size":27091,"ondisk_log_size":27091,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304052,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067788,"statfs":{"total":96636764160,"available":96325414912,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.92100000000000004,"15min":0.90900000000000003},"min":{"1min":0.54700000000000004,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.022},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.91100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.25,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.85199999999999998,"15min":0.85999999999999999},"min":{"1min":0.41199999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0409999999999999,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.341,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1240000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.52900000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2529999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.143},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.45800000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.371,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":1.081}]},{"osd":3,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0149999999999999,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2689999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.0089999999999999},"min":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":0.81899999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.5549999999999999,"5min":3.246,"15min":3.246},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.024,"15min":1.0369999999999999},"min":{"1min":0.53400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.2909999999999999,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.216}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.0249999999999999},"min":{"1min":0.63700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.165,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.046,"5min":1.0389999999999999,"15min":1.0189999999999999},"min":{"1min":0.47199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5229999999999999,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":1.2430000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0840000000000001,"15min":1.069},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4650000000000001,"5min":3.492,"15min":3.492},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.115,"5min":1.123,"15min":1.131},"min":{"1min":0.59899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.577,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.27}]}]},{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312372,"kb_used_data":2536,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059468,"statfs":{"total":96636764160,"available":96316895232,"internally_reserved":0,"allocated":2596864,"data_stored":1250825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1850000000000001,"5min":1.498,"15min":1.498},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.0169999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.86299999999999999,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2729999999999999,"5min":1.413,"15min":1.413},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.95499999999999996,"15min":0.995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.63600000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84099999999999997,"15min":0.85699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.872},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.175,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.73199999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.577,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.109},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1579999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.88400000000000001,"15min":0.95399999999999996},"min":{"1min":0.47699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1200000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94199999999999995,"15min":0.97199999999999998},"min":{"1min":0.48299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.3719999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.96399999999999997,"15min":0.97199999999999998},"min":{"1min":0.56100000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.976,"5min":4.976,"15min":4.976},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.03,"15min":1.0549999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.66400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:10:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.038,"15min":1.0600000000000001},"min":{"1min":0.55000000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":3.4319999999999999,"5min":3.4319999999999999,"15min":3.4319999999999999},"last":1.21},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.101,"15min":1.1419999999999999},"min":{"1min":0.57399999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.8799999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":0.96099999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509494,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304116,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067724,"statfs":{"total":96636764160,"available":96325349376,"internally_reserved":0,"allocated":2654208,"data_stored":1841193,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97299999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.78700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.77700000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.85399999999999998,"15min":0.88500000000000001},"min":{"1min":0.48899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.048,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.90300000000000002,"15min":0.95499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0900000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.94199999999999995,"15min":0.92300000000000004},"min":{"1min":0.46500000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1479999999999999,"5min":10.999000000000001,"15min":10.999000000000001},"last":1.028},{"interface":"front","average":{"1min":0.88100000000000001,"5min":1.0469999999999999,"15min":1.05},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.276,"5min":11.018000000000001,"15min":11.018000000000001},"last":1.1220000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.212,"15min":1.212},"last":1.2649999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.89400000000000002,"15min":0.93799999999999994},"min":{"1min":0.60399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.125,"5min":1.601,"15min":1.601},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.96699999999999997,"15min":0.996},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.244,"5min":1.764,"15min":1.764},"last":1.1859999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.90300000000000002,"15min":0.93799999999999994},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.538,"15min":1.538},"last":1.353},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.999,"15min":1.0580000000000001},"min":{"1min":0.67200000000000004,"5min":0.315,"15min":0.315},"max":{"1min":1.409,"5min":1.573,"15min":1.573},"last":1.23}]},{"osd":7,"last update":"Mon Sep 16 14:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.008,"15min":1.046},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3100000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.3},{"interface":"front","average":{"1min":1.014,"5min":1.0660000000000001,"15min":1.1220000000000001},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3759999999999999,"5min":1.619,"15min":1.619},"last":1.4239999999999999}]}]},{"osd":3,"up_from":83,"seq":356482285600,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309944,"kb_used_data":2536,"kb_used_omap":31,"kb_used_meta":307360,"kb_avail":94061896,"statfs":{"total":96636764160,"available":96319381504,"internally_reserved":0,"allocated":2596864,"data_stored":1250825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31864,"internal_metadata":314737544},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":1,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.36699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":1.468,"15min":1.468},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1459999999999999,"5min":1.258,"15min":1.258},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93200000000000005,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.49199999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.89600000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.119,"5min":1.347,"15min":1.347},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.96399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.85699999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95999999999999996,"15min":0.95299999999999996},"min":{"1min":0.64900000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.0489999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0349999999999999,"15min":1.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.042},"min":{"1min":0.72099999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.454,"5min":1.486,"15min":1.486},"last":0.96099999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.024,"15min":1.024},"min":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2929999999999999,"5min":1.419,"15min":1.419},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.103,"5min":1.0369999999999999,"15min":1.026},"min":{"1min":0.77500000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4139999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.93000000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308208,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063632,"statfs":{"total":96636764160,"available":96321159168,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77500000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.044,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73699999999999999,"15min":0.75900000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.276,"5min":1.454,"15min":1.454},"last":0.74399999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.5,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4330000000000001,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.03},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.244}]},{"osd":5,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.85599999999999998,"15min":0.85199999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4390000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.044},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.96699999999999997},"min":{"1min":0.61099999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.948,"15min":1.948},"last":1.073}]},{"osd":6,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92700000000000005,"15min":0.93200000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.5229999999999999,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.016},{"interface":"front","average":{"1min":1.026,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.415,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.1220000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:09:58 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.98799999999999999,"15min":0.999},"min":{"1min":0.53900000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.623,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.194},{"interface":"front","average":{"1min":1.022,"5min":0.96599999999999997,"15min":0.95799999999999996},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4470000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.96599999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312424,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059416,"statfs":{"total":96636764160,"available":96316841984,"internally_reserved":0,"allocated":2637824,"data_stored":1841193,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.80800000000000005,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.5890000000000004,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.625}]},{"osd":2,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.70299999999999996,"15min":0.745},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.90500000000000003,"5min":1.998,"15min":1.998},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.92200000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:10:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":7.9950000000000001,"5min":7.9950000000000001,"15min":7.9950000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0009999999999999,"5min":1.244,"15min":1.244},"last":0.88100000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85399999999999998,"15min":0.89200000000000002},"min":{"1min":0.44900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.89600000000000002,"15min":0.92600000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1359999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":1.087,"15min":1.1200000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2330000000000001,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.006,"15min":1.0660000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.1319999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.077,"15min":1.097},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0600000000000001,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":1.087},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0429999999999999,"15min":1.0960000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.1899999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.018,"15min":1.0649999999999999},"min":{"1min":0.63800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.865,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.145,"15min":1.149},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.375,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":1.149}]}]},{"osd":0,"up_from":184,"seq":790273982474,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311856,"kb_used_data":2532,"kb_used_omap":125,"kb_used_meta":309186,"kb_avail":94059984,"statfs":{"total":96636764160,"available":96317423616,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":128171,"internal_metadata":316607317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316528,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94055312,"statfs":{"total":96636764160,"available":96312639488,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2809999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.60699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":1.2210000000000001,"15min":1.0369999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.121,"5min":37.805,"15min":37.805},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1819999999999999,"15min":1.008},"min":{"1min":0.439,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.155,"5min":37.857999999999997,"15min":37.857999999999997},"last":1.0169999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.115,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.86299999999999999,"15min":0.86399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.204,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:10:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.68000000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.82899999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":0.94699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.133,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.60999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2829999999999999,"5min":1.611,"15min":1.611},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.95899999999999996,"15min":0.97299999999999998},"min":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3009999999999999,"5min":1.429,"15min":1.429},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.999,"5min":1.018,"15min":1.0309999999999999},"min":{"1min":0.67700000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.278,"5min":1.601,"15min":1.601},"last":0.86899999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:09:57 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68700000000000006,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.355,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0780000000000001,"15min":1.097},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.3819999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29034,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:10:58.018 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.019 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.019 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.019 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.019 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.019 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.019 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.019 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.020 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.021 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.021 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.021 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.021 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.021 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.021 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.021 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.021 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T14:06:52.823835+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.022 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.023 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.023 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.023 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.023 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.023 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.023 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.023 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:10:58.023 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T14:11:18.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:11:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:11:18.337 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:11:18.349 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-16T14:11:16.838444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":588,"num_read":281435,"num_read_kb":918233,"num_write":31188,"num_write_kb":443365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9465,"num_bytes_recovered":336866533,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27729,"ondisk_log_size":27729,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2466960,"kb_used_data":20368,"kb_used_omap":166,"kb_used_meta":2446361,"kb_avail":752507760,"statfs":{"total":773094113280,"available":770567946240,"internally_reserved":0,"allocated":20856832,"data_stored":11187260,"data_compressed":11598,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":170259,"internal_metadata":2505074413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003777"},"pg_stats":[{"pgid":"3.1f","version":"215'1893","reported_seq":7915,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.709373+0000","last_change":"2024-09-16T14:11:04.709287+0000","last_active":"2024-09-16T14:11:04.709373+0000","last_peered":"2024-09-16T14:11:04.709373+0000","last_clean":"2024-09-16T14:11:04.709373+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:11:04.709373+0000","last_undegraded":"2024-09-16T14:11:04.709373+0000","last_fullsized":"2024-09-16T14:11:04.709373+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1893","last_scrub_stamp":"2024-09-16T14:11:04.709013+0000","last_deep_scrub":"215'1893","last_deep_scrub_stamp":"2024-09-16T14:11:04.709013+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.709013+0000","objects_scrubbed":17,"log_size":1893,"log_dups_size":0,"ondisk_log_size":1893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:57.055333+0000","scrub_duration":0.039060215000000002,"objects_trimmed":0,"snaptrim_duration":0.0039454019999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15599,"num_read_kb":19767,"num_write":2973,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"215'819","reported_seq":2756,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:03.719451+0000","last_change":"2024-09-16T14:11:03.719414+0000","last_active":"2024-09-16T14:11:03.719451+0000","last_peered":"2024-09-16T14:11:03.719451+0000","last_clean":"2024-09-16T14:11:03.719451+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:11:03.719451+0000","last_undegraded":"2024-09-16T14:11:03.719451+0000","last_fullsized":"2024-09-16T14:11:03.719451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'819","last_scrub_stamp":"2024-09-16T14:11:03.719268+0000","last_deep_scrub":"215'819","last_deep_scrub_stamp":"2024-09-16T14:11:03.719268+0000","last_clean_scrub_stamp":"2024-09-16T14:11:03.719268+0000","objects_scrubbed":19,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:02.067061+0000","scrub_duration":0.029885353,"objects_trimmed":0,"snaptrim_duration":0.00098036900000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3428,"num_read_kb":21398,"num_write":967,"num_write_kb":11613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"215'670","reported_seq":5558,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:10.632313+0000","last_change":"2024-09-16T14:11:10.632251+0000","last_active":"2024-09-16T14:11:10.632313+0000","last_peered":"2024-09-16T14:11:10.632313+0000","last_clean":"2024-09-16T14:11:10.632313+0000","last_became_active":"2024-09-16T14:10:20.387629+0000","last_became_peered":"2024-09-16T14:10:20.387629+0000","last_unstale":"2024-09-16T14:11:10.632313+0000","last_undegraded":"2024-09-16T14:11:10.632313+0000","last_fullsized":"2024-09-16T14:11:10.632313+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'670","last_scrub_stamp":"2024-09-16T14:11:10.632091+0000","last_deep_scrub":"215'670","last_deep_scrub_stamp":"2024-09-16T14:11:10.632091+0000","last_clean_scrub_stamp":"2024-09-16T14:11:10.632091+0000","objects_scrubbed":17,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:43.042825+0000","scrub_duration":0.025392595,"objects_trimmed":0,"snaptrim_duration":0.0051863780000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7537,"num_read_kb":24768,"num_write":780,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":26063912,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"215'1053","reported_seq":14559,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:56.381546+0000","last_change":"2024-09-16T14:10:56.381505+0000","last_active":"2024-09-16T14:10:56.381546+0000","last_peered":"2024-09-16T14:10:56.381546+0000","last_clean":"2024-09-16T14:10:56.381546+0000","last_became_active":"2024-09-16T14:10:20.385216+0000","last_became_peered":"2024-09-16T14:10:20.385216+0000","last_unstale":"2024-09-16T14:10:56.381546+0000","last_undegraded":"2024-09-16T14:10:56.381546+0000","last_fullsized":"2024-09-16T14:10:56.381546+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1053","last_scrub_stamp":"2024-09-16T14:10:56.381406+0000","last_deep_scrub":"215'1053","last_deep_scrub_stamp":"2024-09-16T14:10:56.381406+0000","last_clean_scrub_stamp":"2024-09-16T14:10:56.381406+0000","objects_scrubbed":22,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:03.029898+0000","scrub_duration":0.028541266999999999,"objects_trimmed":0,"snaptrim_duration":0.0013844,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":21,"num_read":15843,"num_read_kb":26936,"num_write":1369,"num_write_kb":11030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":6143753,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"215'761","reported_seq":8115,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.022986+0000","last_change":"2024-09-16T14:11:04.022919+0000","last_active":"2024-09-16T14:11:04.022986+0000","last_peered":"2024-09-16T14:11:04.022986+0000","last_clean":"2024-09-16T14:11:04.022986+0000","last_became_active":"2024-09-16T14:10:20.387007+0000","last_became_peered":"2024-09-16T14:10:20.387007+0000","last_unstale":"2024-09-16T14:11:04.022986+0000","last_undegraded":"2024-09-16T14:11:04.022986+0000","last_fullsized":"2024-09-16T14:11:04.022986+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'761","last_scrub_stamp":"2024-09-16T14:11:04.022818+0000","last_deep_scrub":"215'761","last_deep_scrub_stamp":"2024-09-16T14:11:04.022818+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.022818+0000","objects_scrubbed":23,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:06.193706+0000","scrub_duration":0.027573283000000001,"objects_trimmed":0,"snaptrim_duration":0.0010460949999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10352,"num_read_kb":26550,"num_write":833,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"215'865","reported_seq":8058,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.429899+0000","last_change":"2024-09-16T14:10:57.429835+0000","last_active":"2024-09-16T14:10:57.429899+0000","last_peered":"2024-09-16T14:10:57.429899+0000","last_clean":"2024-09-16T14:10:57.429899+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:10:57.429899+0000","last_undegraded":"2024-09-16T14:10:57.429899+0000","last_fullsized":"2024-09-16T14:10:57.429899+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'865","last_scrub_stamp":"2024-09-16T14:10:57.429718+0000","last_deep_scrub":"215'865","last_deep_scrub_stamp":"2024-09-16T14:10:57.429718+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.429718+0000","objects_scrubbed":20,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:43:58.881095+0000","scrub_duration":0.029096839999999999,"objects_trimmed":0,"snaptrim_duration":0.0017171580000000001,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":10449,"num_read_kb":28504,"num_write":803,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"215'645","reported_seq":2369,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:07.943968+0000","last_change":"2024-09-16T14:11:07.943932+0000","last_active":"2024-09-16T14:11:07.943968+0000","last_peered":"2024-09-16T14:11:07.943968+0000","last_clean":"2024-09-16T14:11:07.943968+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:11:07.943968+0000","last_undegraded":"2024-09-16T14:11:07.943968+0000","last_fullsized":"2024-09-16T14:11:07.943968+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'645","last_scrub_stamp":"2024-09-16T14:11:07.943829+0000","last_deep_scrub":"215'645","last_deep_scrub_stamp":"2024-09-16T14:11:07.943829+0000","last_clean_scrub_stamp":"2024-09-16T14:11:07.943829+0000","objects_scrubbed":15,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:00:34.772882+0000","scrub_duration":0.026295391000000001,"objects_trimmed":0,"snaptrim_duration":0.001483112,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1924,"num_read_kb":28666,"num_write":709,"num_write_kb":21954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"215'986","reported_seq":6393,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.735723+0000","last_change":"2024-09-16T14:11:01.735693+0000","last_active":"2024-09-16T14:11:01.735723+0000","last_peered":"2024-09-16T14:11:01.735723+0000","last_clean":"2024-09-16T14:11:01.735723+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:11:01.735723+0000","last_undegraded":"2024-09-16T14:11:01.735723+0000","last_fullsized":"2024-09-16T14:11:01.735723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'986","last_scrub_stamp":"2024-09-16T14:11:01.735585+0000","last_deep_scrub":"215'986","last_deep_scrub_stamp":"2024-09-16T14:11:01.735585+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.735585+0000","objects_scrubbed":18,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:55.356183+0000","scrub_duration":0.024480147000000001,"objects_trimmed":0,"snaptrim_duration":0.0050019210000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9238,"num_read_kb":21812,"num_write":1162,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"215'1370","reported_seq":6459,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:02.748860+0000","last_change":"2024-09-16T14:11:02.748834+0000","last_active":"2024-09-16T14:11:02.748860+0000","last_peered":"2024-09-16T14:11:02.748860+0000","last_clean":"2024-09-16T14:11:02.748860+0000","last_became_active":"2024-09-16T14:10:05.123005+0000","last_became_peered":"2024-09-16T14:10:05.123005+0000","last_unstale":"2024-09-16T14:11:02.748860+0000","last_undegraded":"2024-09-16T14:11:02.748860+0000","last_fullsized":"2024-09-16T14:11:02.748860+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1370","last_scrub_stamp":"2024-09-16T14:11:02.748740+0000","last_deep_scrub":"215'1370","last_deep_scrub_stamp":"2024-09-16T14:11:02.748740+0000","last_clean_scrub_stamp":"2024-09-16T14:11:02.748740+0000","objects_scrubbed":19,"log_size":1370,"log_dups_size":0,"ondisk_log_size":1370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:08.089103+0000","scrub_duration":0.031876672000000002,"objects_trimmed":0,"snaptrim_duration":0.0039366710000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8929,"num_read_kb":33272,"num_write":1987,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":4502377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"215'774","reported_seq":13976,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:03.970164+0000","last_change":"2024-09-16T14:11:03.970113+0000","last_active":"2024-09-16T14:11:03.970164+0000","last_peered":"2024-09-16T14:11:03.970164+0000","last_clean":"2024-09-16T14:11:03.970164+0000","last_became_active":"2024-09-16T14:10:05.377635+0000","last_became_peered":"2024-09-16T14:10:05.377635+0000","last_unstale":"2024-09-16T14:11:03.970164+0000","last_undegraded":"2024-09-16T14:11:03.970164+0000","last_fullsized":"2024-09-16T14:11:03.970164+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'774","last_scrub_stamp":"2024-09-16T14:11:03.969989+0000","last_deep_scrub":"215'774","last_deep_scrub_stamp":"2024-09-16T14:11:03.969989+0000","last_clean_scrub_stamp":"2024-09-16T14:11:03.969989+0000","objects_scrubbed":23,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:40.975725+0000","scrub_duration":0.031613308,"objects_trimmed":0,"snaptrim_duration":0.00089383699999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":14525,"num_read_kb":50269,"num_write":689,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":11596768,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"215'601","reported_seq":5226,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:00.892229+0000","last_change":"2024-09-16T14:11:00.892194+0000","last_active":"2024-09-16T14:11:00.892229+0000","last_peered":"2024-09-16T14:11:00.892229+0000","last_clean":"2024-09-16T14:11:00.892229+0000","last_became_active":"2024-09-16T14:10:19.498121+0000","last_became_peered":"2024-09-16T14:10:19.498121+0000","last_unstale":"2024-09-16T14:11:00.892229+0000","last_undegraded":"2024-09-16T14:11:00.892229+0000","last_fullsized":"2024-09-16T14:11:00.892229+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'601","last_scrub_stamp":"2024-09-16T14:11:00.892082+0000","last_deep_scrub":"215'601","last_deep_scrub_stamp":"2024-09-16T14:11:00.892082+0000","last_clean_scrub_stamp":"2024-09-16T14:11:00.892082+0000","objects_scrubbed":25,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:30.076930+0000","scrub_duration":0.035753736000000001,"objects_trimmed":0,"snaptrim_duration":0.002716544,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4971,"num_read_kb":28564,"num_write":500,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":50024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"197'723","reported_seq":7825,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865561+0000","last_change":"2024-09-16T14:10:29.582367+0000","last_active":"2024-09-16T14:10:47.865561+0000","last_peered":"2024-09-16T14:10:47.865561+0000","last_clean":"2024-09-16T14:10:47.865561+0000","last_became_active":"2024-09-16T14:10:06.388650+0000","last_became_peered":"2024-09-16T14:10:06.388650+0000","last_unstale":"2024-09-16T14:10:47.865561+0000","last_undegraded":"2024-09-16T14:10:47.865561+0000","last_fullsized":"2024-09-16T14:10:47.865561+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054744670000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9236,"num_read_kb":38811,"num_write":682,"num_write_kb":10309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":10174643,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865735+0000","last_change":"2024-09-16T14:10:17.556015+0000","last_active":"2024-09-16T14:10:47.865735+0000","last_peered":"2024-09-16T14:10:47.865735+0000","last_clean":"2024-09-16T14:10:47.865735+0000","last_became_active":"2024-09-16T14:10:17.555698+0000","last_became_peered":"2024-09-16T14:10:17.555698+0000","last_unstale":"2024-09-16T14:10:47.865735+0000","last_undegraded":"2024-09-16T14:10:47.865735+0000","last_fullsized":"2024-09-16T14:10:47.865735+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"215'913","reported_seq":6204,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.946914+0000","last_change":"2024-09-16T14:10:57.946822+0000","last_active":"2024-09-16T14:10:57.946914+0000","last_peered":"2024-09-16T14:10:57.946914+0000","last_clean":"2024-09-16T14:10:57.946914+0000","last_became_active":"2024-09-16T14:10:25.385245+0000","last_became_peered":"2024-09-16T14:10:25.385245+0000","last_unstale":"2024-09-16T14:10:57.946914+0000","last_undegraded":"2024-09-16T14:10:57.946914+0000","last_fullsized":"2024-09-16T14:10:57.946914+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'913","last_scrub_stamp":"2024-09-16T14:10:57.946694+0000","last_deep_scrub":"215'913","last_deep_scrub_stamp":"2024-09-16T14:10:57.946694+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.946694+0000","objects_scrubbed":16,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:11.677456+0000","scrub_duration":0.024780823,"objects_trimmed":0,"snaptrim_duration":0.0026361219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5530,"num_read_kb":25811,"num_write":916,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":731,"num_bytes_recovered":15701912,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:06.947533+0000","last_change":"2024-09-16T14:11:06.947388+0000","last_active":"2024-09-16T14:11:06.947533+0000","last_peered":"2024-09-16T14:11:06.947533+0000","last_clean":"2024-09-16T14:11:06.947533+0000","last_became_active":"2024-09-16T14:10:16.471977+0000","last_became_peered":"2024-09-16T14:10:16.471977+0000","last_unstale":"2024-09-16T14:11:06.947533+0000","last_undegraded":"2024-09-16T14:11:06.947533+0000","last_fullsized":"2024-09-16T14:11:06.947533+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:11:06.947138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:11:06.947138+0000","last_clean_scrub_stamp":"2024-09-16T14:11:06.947138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:34.633141+0000","scrub_duration":0.0036792880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"215'991","reported_seq":7760,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.016287+0000","last_change":"2024-09-16T14:11:01.016189+0000","last_active":"2024-09-16T14:11:01.016287+0000","last_peered":"2024-09-16T14:11:01.016287+0000","last_clean":"2024-09-16T14:11:01.016287+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:11:01.016287+0000","last_undegraded":"2024-09-16T14:11:01.016287+0000","last_fullsized":"2024-09-16T14:11:01.016287+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'991","last_scrub_stamp":"2024-09-16T14:11:01.016033+0000","last_deep_scrub":"215'991","last_deep_scrub_stamp":"2024-09-16T14:11:01.016033+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.016033+0000","objects_scrubbed":24,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:02.743229+0000","scrub_duration":0.032062161999999998,"objects_trimmed":0,"snaptrim_duration":0.002259535,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":10392,"num_read_kb":45651,"num_write":1244,"num_write_kb":30465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:58.748481+0000","last_change":"2024-09-16T14:10:58.748412+0000","last_active":"2024-09-16T14:10:58.748481+0000","last_peered":"2024-09-16T14:10:58.748481+0000","last_clean":"2024-09-16T14:10:58.748481+0000","last_became_active":"2024-09-16T14:10:00.102521+0000","last_became_peered":"2024-09-16T14:10:00.102521+0000","last_unstale":"2024-09-16T14:10:58.748481+0000","last_undegraded":"2024-09-16T14:10:58.748481+0000","last_fullsized":"2024-09-16T14:10:58.748481+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:10:58.748188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:10:58.748188+0000","last_clean_scrub_stamp":"2024-09-16T14:10:58.748188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:54.126496+0000","scrub_duration":0.0037129939999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"215'647","reported_seq":3980,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:02.252304+0000","last_change":"2024-09-16T14:11:02.252277+0000","last_active":"2024-09-16T14:11:02.252304+0000","last_peered":"2024-09-16T14:11:02.252304+0000","last_clean":"2024-09-16T14:11:02.252304+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:11:02.252304+0000","last_undegraded":"2024-09-16T14:11:02.252304+0000","last_fullsized":"2024-09-16T14:11:02.252304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'647","last_scrub_stamp":"2024-09-16T14:11:02.252154+0000","last_deep_scrub":"215'647","last_deep_scrub_stamp":"2024-09-16T14:11:02.252154+0000","last_clean_scrub_stamp":"2024-09-16T14:11:02.252154+0000","objects_scrubbed":17,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:29.668999+0000","scrub_duration":0.026147526000000001,"objects_trimmed":0,"snaptrim_duration":0.001012095,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4560,"num_read_kb":16131,"num_write":597,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:59.755353+0000","last_change":"2024-09-16T14:10:59.755275+0000","last_active":"2024-09-16T14:10:59.755353+0000","last_peered":"2024-09-16T14:10:59.755353+0000","last_clean":"2024-09-16T14:10:59.755353+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:10:59.755353+0000","last_undegraded":"2024-09-16T14:10:59.755353+0000","last_fullsized":"2024-09-16T14:10:59.755353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:10:59.755046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:10:59.755046+0000","last_clean_scrub_stamp":"2024-09-16T14:10:59.755046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:47:48.201040+0000","scrub_duration":0.003186361,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"215'792","reported_seq":9836,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.981555+0000","last_change":"2024-09-16T14:11:04.981524+0000","last_active":"2024-09-16T14:11:04.981555+0000","last_peered":"2024-09-16T14:11:04.981555+0000","last_clean":"2024-09-16T14:11:04.981555+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:11:04.981555+0000","last_undegraded":"2024-09-16T14:11:04.981555+0000","last_fullsized":"2024-09-16T14:11:04.981555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'792","last_scrub_stamp":"2024-09-16T14:11:04.981436+0000","last_deep_scrub":"215'792","last_deep_scrub_stamp":"2024-09-16T14:11:04.981436+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.981436+0000","objects_scrubbed":12,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:40.393594+0000","scrub_duration":0.019826035999999998,"objects_trimmed":0,"snaptrim_duration":0.001064281,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":11041,"num_read_kb":12619,"num_write":857,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"215'1434","reported_seq":5266,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.815104+0000","last_change":"2024-09-16T14:10:57.815074+0000","last_active":"2024-09-16T14:10:57.815104+0000","last_peered":"2024-09-16T14:10:57.815104+0000","last_clean":"2024-09-16T14:10:57.815104+0000","last_became_active":"2024-09-16T14:10:19.505692+0000","last_became_peered":"2024-09-16T14:10:19.505692+0000","last_unstale":"2024-09-16T14:10:57.815104+0000","last_undegraded":"2024-09-16T14:10:57.815104+0000","last_fullsized":"2024-09-16T14:10:57.815104+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1434","last_scrub_stamp":"2024-09-16T14:10:57.814968+0000","last_deep_scrub":"215'1434","last_deep_scrub_stamp":"2024-09-16T14:10:57.814968+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.814968+0000","objects_scrubbed":23,"log_size":1434,"log_dups_size":0,"ondisk_log_size":1434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:44:00.620195+0000","scrub_duration":0.032517719,"objects_trimmed":0,"snaptrim_duration":0.0040141910000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":9687,"num_read_kb":19616,"num_write":1641,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"215'3","reported_seq":68,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:07.593489+0000","last_change":"2024-09-16T14:11:07.593404+0000","last_active":"2024-09-16T14:11:07.593489+0000","last_peered":"2024-09-16T14:11:07.593489+0000","last_clean":"2024-09-16T14:11:07.593489+0000","last_became_active":"2024-09-16T14:10:00.104539+0000","last_became_peered":"2024-09-16T14:10:00.104539+0000","last_unstale":"2024-09-16T14:11:07.593489+0000","last_undegraded":"2024-09-16T14:11:07.593489+0000","last_fullsized":"2024-09-16T14:11:07.593489+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'3","last_scrub_stamp":"2024-09-16T14:11:07.593178+0000","last_deep_scrub":"215'3","last_deep_scrub_stamp":"2024-09-16T14:11:07.593178+0000","last_clean_scrub_stamp":"2024-09-16T14:11:07.593178+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:05.249880+0000","scrub_duration":0.0089886010000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"215'1337","reported_seq":6578,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:05.735842+0000","last_change":"2024-09-16T14:11:05.735811+0000","last_active":"2024-09-16T14:11:05.735842+0000","last_peered":"2024-09-16T14:11:05.735842+0000","last_clean":"2024-09-16T14:11:05.735842+0000","last_became_active":"2024-09-16T14:10:26.385527+0000","last_became_peered":"2024-09-16T14:10:26.385527+0000","last_unstale":"2024-09-16T14:11:05.735842+0000","last_undegraded":"2024-09-16T14:11:05.735842+0000","last_fullsized":"2024-09-16T14:11:05.735842+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1337","last_scrub_stamp":"2024-09-16T14:11:05.735690+0000","last_deep_scrub":"215'1337","last_deep_scrub_stamp":"2024-09-16T14:11:05.735690+0000","last_clean_scrub_stamp":"2024-09-16T14:11:05.735690+0000","objects_scrubbed":21,"log_size":1337,"log_dups_size":0,"ondisk_log_size":1337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:31.726396+0000","scrub_duration":0.032108896999999997,"objects_trimmed":0,"snaptrim_duration":0.0051618139999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9488,"num_read_kb":24272,"num_write":1738,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":8428888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.310710+0000","last_change":"2024-09-16T14:11:01.310645+0000","last_active":"2024-09-16T14:11:01.310710+0000","last_peered":"2024-09-16T14:11:01.310710+0000","last_clean":"2024-09-16T14:11:01.310710+0000","last_became_active":"2024-09-16T14:10:21.387173+0000","last_became_peered":"2024-09-16T14:10:21.387173+0000","last_unstale":"2024-09-16T14:11:01.310710+0000","last_undegraded":"2024-09-16T14:11:01.310710+0000","last_fullsized":"2024-09-16T14:11:01.310710+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:11:01.310374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:11:01.310374+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.310374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:06.388664+0000","scrub_duration":0.003307089,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"215'843","reported_seq":7064,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.281157+0000","last_change":"2024-09-16T14:10:57.281110+0000","last_active":"2024-09-16T14:10:57.281157+0000","last_peered":"2024-09-16T14:10:57.281157+0000","last_clean":"2024-09-16T14:10:57.281157+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:10:57.281157+0000","last_undegraded":"2024-09-16T14:10:57.281157+0000","last_fullsized":"2024-09-16T14:10:57.281157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'843","last_scrub_stamp":"2024-09-16T14:10:57.281004+0000","last_deep_scrub":"215'843","last_deep_scrub_stamp":"2024-09-16T14:10:57.281004+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.281004+0000","objects_scrubbed":23,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:12.179239+0000","scrub_duration":0.032072307000000001,"objects_trimmed":0,"snaptrim_duration":0.001233734,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":8272,"num_read_kb":21183,"num_write":947,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865309+0000","last_change":"2024-09-16T14:10:00.104224+0000","last_active":"2024-09-16T14:10:47.865309+0000","last_peered":"2024-09-16T14:10:47.865309+0000","last_clean":"2024-09-16T14:10:47.865309+0000","last_became_active":"2024-09-16T14:10:00.104003+0000","last_became_peered":"2024-09-16T14:10:00.104003+0000","last_unstale":"2024-09-16T14:10:47.865309+0000","last_undegraded":"2024-09-16T14:10:47.865309+0000","last_fullsized":"2024-09-16T14:10:47.865309+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:06:59.453408+0000","last_clean_scrub_stamp":"2024-09-16T14:06:59.453408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"215'681","reported_seq":15134,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:12.881710+0000","last_change":"2024-09-16T14:11:12.881592+0000","last_active":"2024-09-16T14:11:12.881710+0000","last_peered":"2024-09-16T14:11:12.881710+0000","last_clean":"2024-09-16T14:11:12.881710+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:11:12.881710+0000","last_undegraded":"2024-09-16T14:11:12.881710+0000","last_fullsized":"2024-09-16T14:11:12.881710+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'681","last_scrub_stamp":"2024-09-16T14:11:12.881388+0000","last_deep_scrub":"215'681","last_deep_scrub_stamp":"2024-09-16T14:11:12.881388+0000","last_clean_scrub_stamp":"2024-09-16T14:11:12.881388+0000","objects_scrubbed":14,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:47.689768+0000","scrub_duration":0.021531431,"objects_trimmed":0,"snaptrim_duration":0.0020715640000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":14807,"num_read_kb":52699,"num_write":621,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"215'2","reported_seq":68,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.743051+0000","last_change":"2024-09-16T14:10:57.743012+0000","last_active":"2024-09-16T14:10:57.743051+0000","last_peered":"2024-09-16T14:10:57.743051+0000","last_clean":"2024-09-16T14:10:57.743051+0000","last_became_active":"2024-09-16T14:10:00.105302+0000","last_became_peered":"2024-09-16T14:10:00.105302+0000","last_unstale":"2024-09-16T14:10:57.743051+0000","last_undegraded":"2024-09-16T14:10:57.743051+0000","last_fullsized":"2024-09-16T14:10:57.743051+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'2","last_scrub_stamp":"2024-09-16T14:10:57.742863+0000","last_deep_scrub":"215'2","last_deep_scrub_stamp":"2024-09-16T14:10:57.742863+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.742863+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:11.383869+0000","scrub_duration":0.0043886649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"215'74","reported_seq":116,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.441740+0000","last_change":"2024-09-16T14:11:01.441669+0000","last_active":"2024-09-16T14:11:01.441740+0000","last_peered":"2024-09-16T14:11:01.441740+0000","last_clean":"2024-09-16T14:11:01.441740+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:11:01.441740+0000","last_undegraded":"2024-09-16T14:11:01.441740+0000","last_fullsized":"2024-09-16T14:11:01.441740+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'74","last_scrub_stamp":"2024-09-16T14:11:01.441487+0000","last_deep_scrub":"215'74","last_deep_scrub_stamp":"2024-09-16T14:11:01.441487+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.441487+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:52:47.519144+0000","scrub_duration":0.010507404999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"215'727","reported_seq":4321,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:59.994214+0000","last_change":"2024-09-16T14:10:59.994163+0000","last_active":"2024-09-16T14:10:59.994214+0000","last_peered":"2024-09-16T14:10:59.994214+0000","last_clean":"2024-09-16T14:10:59.994214+0000","last_became_active":"2024-09-16T14:10:26.384950+0000","last_became_peered":"2024-09-16T14:10:26.384950+0000","last_unstale":"2024-09-16T14:10:59.994214+0000","last_undegraded":"2024-09-16T14:10:59.994214+0000","last_fullsized":"2024-09-16T14:10:59.994214+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'727","last_scrub_stamp":"2024-09-16T14:10:59.994060+0000","last_deep_scrub":"215'727","last_deep_scrub_stamp":"2024-09-16T14:10:59.994060+0000","last_clean_scrub_stamp":"2024-09-16T14:10:59.994060+0000","objects_scrubbed":23,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:43.925119+0000","scrub_duration":0.028713651999999999,"objects_trimmed":0,"snaptrim_duration":0.00088242400000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4245,"num_read_kb":11739,"num_write":722,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":2087145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"215'642","reported_seq":6992,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.721211+0000","last_change":"2024-09-16T14:11:04.721181+0000","last_active":"2024-09-16T14:11:04.721211+0000","last_peered":"2024-09-16T14:11:04.721211+0000","last_clean":"2024-09-16T14:11:04.721211+0000","last_became_active":"2024-09-16T14:10:08.396086+0000","last_became_peered":"2024-09-16T14:10:08.396086+0000","last_unstale":"2024-09-16T14:11:04.721211+0000","last_undegraded":"2024-09-16T14:11:04.721211+0000","last_fullsized":"2024-09-16T14:11:04.721211+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'642","last_scrub_stamp":"2024-09-16T14:11:04.721065+0000","last_deep_scrub":"215'642","last_deep_scrub_stamp":"2024-09-16T14:11:04.721065+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.721065+0000","objects_scrubbed":19,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:39.505932+0000","scrub_duration":0.023087542999999999,"objects_trimmed":0,"snaptrim_duration":0.00321149,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6657,"num_read_kb":16604,"num_write":570,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":10317335,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"215'871","reported_seq":6931,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:05.691498+0000","last_change":"2024-09-16T14:11:05.691461+0000","last_active":"2024-09-16T14:11:05.691498+0000","last_peered":"2024-09-16T14:11:05.691498+0000","last_clean":"2024-09-16T14:11:05.691498+0000","last_became_active":"2024-09-16T14:10:07.390964+0000","last_became_peered":"2024-09-16T14:10:07.390964+0000","last_unstale":"2024-09-16T14:11:05.691498+0000","last_undegraded":"2024-09-16T14:11:05.691498+0000","last_fullsized":"2024-09-16T14:11:05.691498+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'871","last_scrub_stamp":"2024-09-16T14:11:05.691316+0000","last_deep_scrub":"215'871","last_deep_scrub_stamp":"2024-09-16T14:11:05.691316+0000","last_clean_scrub_stamp":"2024-09-16T14:11:05.691316+0000","objects_scrubbed":21,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:09.065273+0000","scrub_duration":0.029089304,"objects_trimmed":0,"snaptrim_duration":0.0041712620000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":7231,"num_read_kb":39110,"num_write":775,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":49748621,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"215'770","reported_seq":5216,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.334984+0000","last_change":"2024-09-16T14:11:04.334949+0000","last_active":"2024-09-16T14:11:04.334984+0000","last_peered":"2024-09-16T14:11:04.334984+0000","last_clean":"2024-09-16T14:11:04.334984+0000","last_became_active":"2024-09-16T14:10:24.410718+0000","last_became_peered":"2024-09-16T14:10:24.410718+0000","last_unstale":"2024-09-16T14:11:04.334984+0000","last_undegraded":"2024-09-16T14:11:04.334984+0000","last_fullsized":"2024-09-16T14:11:04.334984+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'770","last_scrub_stamp":"2024-09-16T14:11:04.334853+0000","last_deep_scrub":"215'770","last_deep_scrub_stamp":"2024-09-16T14:11:04.334853+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.334853+0000","objects_scrubbed":23,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:11:33.852245+0000","scrub_duration":0.032357058000000001,"objects_trimmed":0,"snaptrim_duration":0.00064224900000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7094,"num_read_kb":44664,"num_write":801,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":19444502,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"215'645","reported_seq":8169,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:59.892384+0000","last_change":"2024-09-16T14:10:59.892354+0000","last_active":"2024-09-16T14:10:59.892384+0000","last_peered":"2024-09-16T14:10:59.892384+0000","last_clean":"2024-09-16T14:10:59.892384+0000","last_became_active":"2024-09-16T14:10:25.384634+0000","last_became_peered":"2024-09-16T14:10:25.384634+0000","last_unstale":"2024-09-16T14:10:59.892384+0000","last_undegraded":"2024-09-16T14:10:59.892384+0000","last_fullsized":"2024-09-16T14:10:59.892384+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'645","last_scrub_stamp":"2024-09-16T14:10:59.892274+0000","last_deep_scrub":"215'645","last_deep_scrub_stamp":"2024-09-16T14:10:59.892274+0000","last_clean_scrub_stamp":"2024-09-16T14:10:59.892274+0000","objects_scrubbed":24,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:07.283676+0000","scrub_duration":0.029250721,"objects_trimmed":0,"snaptrim_duration":0.0034031370000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7377,"num_read_kb":27693,"num_write":494,"num_write_kb":13033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":23836787,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"197'496","reported_seq":2795,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865603+0000","last_change":"2024-09-16T14:10:29.581839+0000","last_active":"2024-09-16T14:10:47.865603+0000","last_peered":"2024-09-16T14:10:47.865603+0000","last_clean":"2024-09-16T14:10:47.865603+0000","last_became_active":"2024-09-16T14:10:04.123286+0000","last_became_peered":"2024-09-16T14:10:04.123286+0000","last_unstale":"2024-09-16T14:10:47.865603+0000","last_undegraded":"2024-09-16T14:10:47.865603+0000","last_fullsized":"2024-09-16T14:10:47.865603+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054803270000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":25169,"num_write":396,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":20371845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"215'524","reported_seq":7424,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:02.941800+0000","last_change":"2024-09-16T14:11:02.941748+0000","last_active":"2024-09-16T14:11:02.941800+0000","last_peered":"2024-09-16T14:11:02.941800+0000","last_clean":"2024-09-16T14:11:02.941800+0000","last_became_active":"2024-09-16T14:10:16.487713+0000","last_became_peered":"2024-09-16T14:10:16.487713+0000","last_unstale":"2024-09-16T14:11:02.941800+0000","last_undegraded":"2024-09-16T14:11:02.941800+0000","last_fullsized":"2024-09-16T14:11:02.941800+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'524","last_scrub_stamp":"2024-09-16T14:11:02.941570+0000","last_deep_scrub":"215'524","last_deep_scrub_stamp":"2024-09-16T14:11:02.941570+0000","last_clean_scrub_stamp":"2024-09-16T14:11:02.941570+0000","objects_scrubbed":14,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:12.209788+0000","scrub_duration":0.020611933999999998,"objects_trimmed":0,"snaptrim_duration":0.00072767700000000003,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7823,"num_read_kb":31641,"num_write":482,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"215'1017","reported_seq":6924,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.919365+0000","last_change":"2024-09-16T14:11:01.919302+0000","last_active":"2024-09-16T14:11:01.919365+0000","last_peered":"2024-09-16T14:11:01.919365+0000","last_clean":"2024-09-16T14:11:01.919365+0000","last_became_active":"2024-09-16T14:10:19.501036+0000","last_became_peered":"2024-09-16T14:10:19.501036+0000","last_unstale":"2024-09-16T14:11:01.919365+0000","last_undegraded":"2024-09-16T14:11:01.919365+0000","last_fullsized":"2024-09-16T14:11:01.919365+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1017","last_scrub_stamp":"2024-09-16T14:11:01.919152+0000","last_deep_scrub":"215'1017","last_deep_scrub_stamp":"2024-09-16T14:11:01.919152+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.919152+0000","objects_scrubbed":14,"log_size":1017,"log_dups_size":0,"ondisk_log_size":1017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:25.383116+0000","scrub_duration":0.025920904000000002,"objects_trimmed":0,"snaptrim_duration":0.0041967539999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10106,"num_read_kb":24848,"num_write":1204,"num_write_kb":15655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":8219627,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"215'638","reported_seq":12664,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:59.401294+0000","last_change":"2024-09-16T14:10:59.401266+0000","last_active":"2024-09-16T14:10:59.401294+0000","last_peered":"2024-09-16T14:10:59.401294+0000","last_clean":"2024-09-16T14:10:59.401294+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:10:59.401294+0000","last_undegraded":"2024-09-16T14:10:59.401294+0000","last_fullsized":"2024-09-16T14:10:59.401294+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'638","last_scrub_stamp":"2024-09-16T14:10:59.401187+0000","last_deep_scrub":"215'638","last_deep_scrub_stamp":"2024-09-16T14:10:59.401187+0000","last_clean_scrub_stamp":"2024-09-16T14:10:59.401187+0000","objects_scrubbed":13,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:46.492819+0000","scrub_duration":0.021501461999999999,"objects_trimmed":0,"snaptrim_duration":0.001114802,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13307,"num_read_kb":24046,"num_write":748,"num_write_kb":16341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"215'1001","reported_seq":6269,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:09.614897+0000","last_change":"2024-09-16T14:11:09.614839+0000","last_active":"2024-09-16T14:11:09.614897+0000","last_peered":"2024-09-16T14:11:09.614897+0000","last_clean":"2024-09-16T14:11:09.614897+0000","last_became_active":"2024-09-16T14:10:08.399369+0000","last_became_peered":"2024-09-16T14:10:08.399369+0000","last_unstale":"2024-09-16T14:11:09.614897+0000","last_undegraded":"2024-09-16T14:11:09.614897+0000","last_fullsized":"2024-09-16T14:11:09.614897+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1001","last_scrub_stamp":"2024-09-16T14:11:09.614716+0000","last_deep_scrub":"215'1001","last_deep_scrub_stamp":"2024-09-16T14:11:09.614716+0000","last_clean_scrub_stamp":"2024-09-16T14:11:09.614716+0000","objects_scrubbed":15,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:24:48.420756+0000","scrub_duration":0.022699634999999999,"objects_trimmed":0,"snaptrim_duration":0.0036375169999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6839,"num_read_kb":16656,"num_write":1300,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":7816767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"215'768","reported_seq":6864,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:06.209704+0000","last_change":"2024-09-16T14:11:06.209670+0000","last_active":"2024-09-16T14:11:06.209704+0000","last_peered":"2024-09-16T14:11:06.209704+0000","last_clean":"2024-09-16T14:11:06.209704+0000","last_became_active":"2024-09-16T14:10:05.121914+0000","last_became_peered":"2024-09-16T14:10:05.121914+0000","last_unstale":"2024-09-16T14:11:06.209704+0000","last_undegraded":"2024-09-16T14:11:06.209704+0000","last_fullsized":"2024-09-16T14:11:06.209704+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'768","last_scrub_stamp":"2024-09-16T14:11:06.209574+0000","last_deep_scrub":"215'768","last_deep_scrub_stamp":"2024-09-16T14:11:06.209574+0000","last_clean_scrub_stamp":"2024-09-16T14:11:06.209574+0000","objects_scrubbed":27,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:38.550773+0000","scrub_duration":0.042191224999999999,"objects_trimmed":0,"snaptrim_duration":0.0025226160000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7908,"num_read_kb":56867,"num_write":705,"num_write_kb":31085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":28924045,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"193'753","reported_seq":7846,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:47.865195+0000","last_change":"2024-09-16T14:10:29.580755+0000","last_active":"2024-09-16T14:10:47.865195+0000","last_peered":"2024-09-16T14:10:47.865195+0000","last_clean":"2024-09-16T14:10:47.865195+0000","last_became_active":"2024-09-16T14:10:21.385616+0000","last_became_peered":"2024-09-16T14:10:21.385616+0000","last_unstale":"2024-09-16T14:10:47.865195+0000","last_undegraded":"2024-09-16T14:10:47.865195+0000","last_fullsized":"2024-09-16T14:10:47.865195+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:07:08.513326+0000","last_clean_scrub_stamp":"2024-09-16T14:07:08.513326+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047871169999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8482,"num_read_kb":31842,"num_write":873,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":41601355,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":588,"num_read":281369,"num_read_kb":918178,"num_write":31085,"num_write_kb":441735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9458,"num_bytes_recovered":336276108,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37982,"internal_metadata":0},"log_size":27650,"ondisk_log_size":27650,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509498,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304044,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067796,"statfs":{"total":96636764160,"available":96325423104,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.92100000000000004,"15min":0.90900000000000003},"min":{"1min":0.54700000000000004,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.91100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.25,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.071}]},{"osd":1,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83599999999999997,"15min":0.85499999999999998},"min":{"1min":0.51200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2210000000000001,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.84299999999999997,"15min":0.84199999999999997},"min":{"1min":0.57399999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.24,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.258,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.55500000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1839999999999999,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0149999999999999,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2689999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.0089999999999999},"min":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.044}]},{"osd":4,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.60499999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3080000000000001,"5min":3.246,"15min":3.246},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.60699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.526,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.012,"15min":1.02},"min":{"1min":0.63300000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.351,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.0669999999999999,"15min":1.0289999999999999},"min":{"1min":0.505,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5840000000000001,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":1.1719999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.107,"15min":1.077},"min":{"1min":0.749,"5min":0.439,"15min":0.439},"max":{"1min":1.6990000000000001,"5min":3.492,"15min":3.492},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.141,"15min":1.137},"min":{"1min":0.80600000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8169999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.24}]}]},{"osd":6,"up_from":15,"seq":64424509497,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312368,"kb_used_data":2536,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059472,"statfs":{"total":96636764160,"available":96316899328,"internally_reserved":0,"allocated":2596864,"data_stored":1250825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1850000000000001,"5min":1.498,"15min":1.498},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.93000000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.2949999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.84899999999999998,"15min":0.91100000000000003},"min":{"1min":0.49099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3109999999999999,"5min":1.413,"15min":1.413},"last":1.097},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.98399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.419,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":0.88200000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.84199999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":2.778,"15min":2.778},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82099999999999995,"15min":0.86899999999999999},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.236,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.95799999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.577,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.1479999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88200000000000001,"15min":0.88},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.226}]},{"osd":4,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86299999999999999,"15min":0.94699999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.272,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.97099999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3839999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":1.0189999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94799999999999995,"15min":0.96599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1930000000000001,"5min":4.976,"15min":4.976},"last":1.1950000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.98599999999999999,"15min":1.04},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.113,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":1.0649999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0409999999999999,"15min":1.0609999999999999},"min":{"1min":0.54400000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4770000000000001,"5min":3.4319999999999999,"15min":3.4319999999999999},"last":1.349},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0649999999999999,"15min":1.1299999999999999},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3109999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.1719999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509498,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308264,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063576,"statfs":{"total":96636764160,"available":96321101824,"internally_reserved":0,"allocated":2654208,"data_stored":1841193,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97299999999999998,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.109},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77600000000000002,"15min":0.78700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.101,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.85999999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1970000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.88700000000000001,"15min":0.94999999999999996},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.258,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1}]},{"osd":2,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93000000000000005,"15min":0.91900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.143,"5min":10.999000000000001,"15min":10.999000000000001},"last":1.075},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0249999999999999,"15min":1.0429999999999999},"min":{"1min":0.504,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.488,"5min":11.018000000000001,"15min":11.018000000000001},"last":1.177}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.212,"15min":1.212},"last":0.82899999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.87,"15min":0.93000000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.111,"5min":1.601,"15min":1.601},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96899999999999997,"15min":0.997},"min":{"1min":0.501,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4199999999999999,"5min":1.764,"15min":1.764},"last":1.2030000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.024,"5min":0.98599999999999999,"15min":1.054},"min":{"1min":0.47299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.3260000000000001,"5min":1.573,"15min":1.573},"last":1.044}]},{"osd":7,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0109999999999999,"15min":1.0469999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.0760000000000001,"15min":1.125},"min":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.246}]}]},{"osd":3,"up_from":83,"seq":356482285605,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309936,"kb_used_data":2536,"kb_used_omap":31,"kb_used_meta":307360,"kb_avail":94061904,"statfs":{"total":96636764160,"available":96319389696,"internally_reserved":0,"allocated":2596864,"data_stored":1250825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31864,"internal_metadata":314737544},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.69499999999999995}]},{"osd":1,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.36699999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85899999999999999,"5min":1.468,"15min":1.468},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1459999999999999,"5min":1.258,"15min":1.258},"last":0.79700000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.74299999999999999,"15min":0.75700000000000001},"min":{"1min":0.48999999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93200000000000005,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77600000000000002,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0960000000000001,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.97299999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.88,"15min":0.89600000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.119,"5min":1.347,"15min":1.347},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.95899999999999996,"15min":0.96399999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.127,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.91800000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90400000000000003,"15min":0.89900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95999999999999996,"15min":0.95299999999999996},"min":{"1min":0.64900000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.0920000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0349999999999999,"15min":1.0329999999999999},"min":{"1min":0.80200000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.048,"15min":1.042},"min":{"1min":0.72099999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.454,"5min":1.486,"15min":1.486},"last":1.0680000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.024,"15min":1.024},"min":{"1min":0.66500000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2929999999999999,"5min":1.419,"15min":1.419},"last":1.196},{"interface":"front","average":{"1min":1.103,"5min":1.0369999999999999,"15min":1.026},"min":{"1min":0.77500000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4139999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.177}]}]},{"osd":2,"up_from":15,"seq":64424509499,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308204,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063636,"statfs":{"total":96636764160,"available":96321163264,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.752,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82499999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.69599999999999995,"15min":0.71499999999999997},"min":{"1min":0.41999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.069},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74299999999999999,"15min":0.76100000000000001},"min":{"1min":0.375,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.83799999999999997}]},{"osd":3,"last update":"Mon Sep 16 14:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.5,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.999}]},{"osd":4,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.89100000000000001,"15min":0.877},"min":{"1min":0.77400000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.871,"15min":0.85399999999999998},"min":{"1min":0.59299999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.91400000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":0.90400000000000003,"15min":0.86799999999999999},"min":{"1min":0.63200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.278},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.97799999999999998,"15min":0.97599999999999998},"min":{"1min":0.68700000000000006,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.4750000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.68200000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.49,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.67800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.4099999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.64900000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.0129999999999999,"15min":0.97399999999999998},"min":{"1min":0.78700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.327}]}]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312408,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059432,"statfs":{"total":96636764160,"available":96316858368,"internally_reserved":0,"allocated":2637824,"data_stored":1841193,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.85799999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.80800000000000005,"15min":0.78600000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.5890000000000004,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.63600000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.74099999999999999},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63500000000000001,"15min":0.66900000000000004},"min":{"1min":0.35799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1339999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:10:27 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.41699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":7.9950000000000001,"5min":7.9950000000000001,"15min":7.9950000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0009999999999999,"5min":1.244,"15min":1.244},"last":0.80400000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84199999999999997,"15min":0.88800000000000001},"min":{"1min":0.47099999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.103,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.89900000000000002,"15min":0.92700000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.141}]},{"osd":5,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.0329999999999999,"15min":1.1020000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.272,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.028,"5min":0.99199999999999999,"15min":1.0620000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.214}]},{"osd":6,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.032,"15min":1.0820000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.105,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.0349999999999999,"15min":1.093},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.252}]},{"osd":7,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.056},"min":{"1min":0.56999999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.403,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.1200000000000001,"15min":1.1399999999999999},"min":{"1min":0.67900000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.512,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":1.1859999999999999}]}]},{"osd":0,"up_from":184,"seq":790273982479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311852,"kb_used_data":2532,"kb_used_omap":125,"kb_used_meta":309186,"kb_avail":94059988,"statfs":{"total":96636764160,"available":96317427712,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":128171,"internal_metadata":316607317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542202,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299884,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071956,"statfs":{"total":96636764160,"available":96329682944,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2809999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.149},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.1739999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.208,"15min":1.032},"min":{"1min":0.50800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.329,"5min":37.805,"15min":37.805},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.177,"15min":1.0069999999999999},"min":{"1min":0.58899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.149,"5min":37.857999999999997,"15min":37.857999999999997},"last":1.2689999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.85399999999999998,"15min":0.86299999999999999},"min":{"1min":0.55100000000000005,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3460000000000001,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57999999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.409,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.038}]},{"osd":3,"last update":"Mon Sep 16 14:10:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.68000000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.0740000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.92600000000000005,"15min":0.94399999999999995},"min":{"1min":0.38200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.262,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92100000000000004,"15min":0.94099999999999995},"min":{"1min":0.44600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.385,"5min":1.611,"15min":1.611},"last":1}]},{"osd":6,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.95499999999999996,"15min":0.97199999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.224,"5min":1.429,"15min":1.429},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0269999999999999,"15min":1.034},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.601,"15min":1.601},"last":1.24}]},{"osd":7,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0289999999999999,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3480000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.081,"15min":1.0980000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.464,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.2030000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29034,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:11:18.353 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:11:18.353 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:11:18.354 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T14:06:59.453408+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=6, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:11:18.354 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:11:18.354 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-16T14:07:08.513326+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=7, tm_sec=8, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=10, tm_sec=52, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:11:18.354 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T14:11:38.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:11:38.654 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-09-16T14:11:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-09-16T14:11:38.666 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-16T14:11:36.844766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":588,"num_read":281435,"num_read_kb":918233,"num_write":31188,"num_write_kb":443365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9465,"num_bytes_recovered":336866533,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27760,"ondisk_log_size":27760,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2466960,"kb_used_data":20368,"kb_used_omap":166,"kb_used_meta":2446361,"kb_avail":752507760,"statfs":{"total":773094113280,"available":770567946240,"internally_reserved":0,"allocated":20856832,"data_stored":11187260,"data_compressed":11598,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":170259,"internal_metadata":2505074413},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003687"},"pg_stats":[{"pgid":"3.1f","version":"215'1893","reported_seq":7915,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.709373+0000","last_change":"2024-09-16T14:11:04.709287+0000","last_active":"2024-09-16T14:11:04.709373+0000","last_peered":"2024-09-16T14:11:04.709373+0000","last_clean":"2024-09-16T14:11:04.709373+0000","last_became_active":"2024-09-16T14:09:14.388452+0000","last_became_peered":"2024-09-16T14:09:14.388452+0000","last_unstale":"2024-09-16T14:11:04.709373+0000","last_undegraded":"2024-09-16T14:11:04.709373+0000","last_fullsized":"2024-09-16T14:11:04.709373+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1893","last_scrub_stamp":"2024-09-16T14:11:04.709013+0000","last_deep_scrub":"215'1893","last_deep_scrub_stamp":"2024-09-16T14:11:04.709013+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.709013+0000","objects_scrubbed":17,"log_size":1893,"log_dups_size":0,"ondisk_log_size":1893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:57.055333+0000","scrub_duration":0.039060215000000002,"objects_trimmed":0,"snaptrim_duration":0.0039454019999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15599,"num_read_kb":19767,"num_write":2973,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":6142255,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"215'819","reported_seq":2756,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:03.719451+0000","last_change":"2024-09-16T14:11:03.719414+0000","last_active":"2024-09-16T14:11:03.719451+0000","last_peered":"2024-09-16T14:11:03.719451+0000","last_clean":"2024-09-16T14:11:03.719451+0000","last_became_active":"2024-09-16T14:07:09.537764+0000","last_became_peered":"2024-09-16T14:07:09.537764+0000","last_unstale":"2024-09-16T14:11:03.719451+0000","last_undegraded":"2024-09-16T14:11:03.719451+0000","last_fullsized":"2024-09-16T14:11:03.719451+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'819","last_scrub_stamp":"2024-09-16T14:11:03.719268+0000","last_deep_scrub":"215'819","last_deep_scrub_stamp":"2024-09-16T14:11:03.719268+0000","last_clean_scrub_stamp":"2024-09-16T14:11:03.719268+0000","objects_scrubbed":19,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:02.067061+0000","scrub_duration":0.029885353,"objects_trimmed":0,"snaptrim_duration":0.00098036900000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3428,"num_read_kb":21398,"num_write":967,"num_write_kb":11613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"215'670","reported_seq":5558,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:10.632313+0000","last_change":"2024-09-16T14:11:10.632251+0000","last_active":"2024-09-16T14:11:10.632313+0000","last_peered":"2024-09-16T14:11:10.632313+0000","last_clean":"2024-09-16T14:11:10.632313+0000","last_became_active":"2024-09-16T14:10:20.387629+0000","last_became_peered":"2024-09-16T14:10:20.387629+0000","last_unstale":"2024-09-16T14:11:10.632313+0000","last_undegraded":"2024-09-16T14:11:10.632313+0000","last_fullsized":"2024-09-16T14:11:10.632313+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'670","last_scrub_stamp":"2024-09-16T14:11:10.632091+0000","last_deep_scrub":"215'670","last_deep_scrub_stamp":"2024-09-16T14:11:10.632091+0000","last_clean_scrub_stamp":"2024-09-16T14:11:10.632091+0000","objects_scrubbed":17,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:43.042825+0000","scrub_duration":0.025392595,"objects_trimmed":0,"snaptrim_duration":0.0051863780000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":7537,"num_read_kb":24768,"num_write":780,"num_write_kb":15707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":878,"num_bytes_recovered":26063912,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"215'1053","reported_seq":14559,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:56.381546+0000","last_change":"2024-09-16T14:10:56.381505+0000","last_active":"2024-09-16T14:10:56.381546+0000","last_peered":"2024-09-16T14:10:56.381546+0000","last_clean":"2024-09-16T14:10:56.381546+0000","last_became_active":"2024-09-16T14:10:20.385216+0000","last_became_peered":"2024-09-16T14:10:20.385216+0000","last_unstale":"2024-09-16T14:10:56.381546+0000","last_undegraded":"2024-09-16T14:10:56.381546+0000","last_fullsized":"2024-09-16T14:10:56.381546+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1053","last_scrub_stamp":"2024-09-16T14:10:56.381406+0000","last_deep_scrub":"215'1053","last_deep_scrub_stamp":"2024-09-16T14:10:56.381406+0000","last_clean_scrub_stamp":"2024-09-16T14:10:56.381406+0000","objects_scrubbed":22,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:03.029898+0000","scrub_duration":0.028541266999999999,"objects_trimmed":0,"snaptrim_duration":0.0013844,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":21,"num_read":15843,"num_read_kb":26936,"num_write":1369,"num_write_kb":11030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":6143753,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"215'761","reported_seq":8115,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.022986+0000","last_change":"2024-09-16T14:11:04.022919+0000","last_active":"2024-09-16T14:11:04.022986+0000","last_peered":"2024-09-16T14:11:04.022986+0000","last_clean":"2024-09-16T14:11:04.022986+0000","last_became_active":"2024-09-16T14:10:20.387007+0000","last_became_peered":"2024-09-16T14:10:20.387007+0000","last_unstale":"2024-09-16T14:11:04.022986+0000","last_undegraded":"2024-09-16T14:11:04.022986+0000","last_fullsized":"2024-09-16T14:11:04.022986+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'761","last_scrub_stamp":"2024-09-16T14:11:04.022818+0000","last_deep_scrub":"215'761","last_deep_scrub_stamp":"2024-09-16T14:11:04.022818+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.022818+0000","objects_scrubbed":23,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:06.193706+0000","scrub_duration":0.027573283000000001,"objects_trimmed":0,"snaptrim_duration":0.0010460949999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10352,"num_read_kb":26550,"num_write":833,"num_write_kb":8180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"215'865","reported_seq":8058,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.429899+0000","last_change":"2024-09-16T14:10:57.429835+0000","last_active":"2024-09-16T14:10:57.429899+0000","last_peered":"2024-09-16T14:10:57.429899+0000","last_clean":"2024-09-16T14:10:57.429899+0000","last_became_active":"2024-09-16T14:09:14.384784+0000","last_became_peered":"2024-09-16T14:09:14.384784+0000","last_unstale":"2024-09-16T14:10:57.429899+0000","last_undegraded":"2024-09-16T14:10:57.429899+0000","last_fullsized":"2024-09-16T14:10:57.429899+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'865","last_scrub_stamp":"2024-09-16T14:10:57.429718+0000","last_deep_scrub":"215'865","last_deep_scrub_stamp":"2024-09-16T14:10:57.429718+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.429718+0000","objects_scrubbed":20,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:43:58.881095+0000","scrub_duration":0.029096839999999999,"objects_trimmed":0,"snaptrim_duration":0.0017171580000000001,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":10449,"num_read_kb":28504,"num_write":803,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":6617381,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"215'645","reported_seq":2369,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:07.943968+0000","last_change":"2024-09-16T14:11:07.943932+0000","last_active":"2024-09-16T14:11:07.943968+0000","last_peered":"2024-09-16T14:11:07.943968+0000","last_clean":"2024-09-16T14:11:07.943968+0000","last_became_active":"2024-09-16T14:09:14.385029+0000","last_became_peered":"2024-09-16T14:09:14.385029+0000","last_unstale":"2024-09-16T14:11:07.943968+0000","last_undegraded":"2024-09-16T14:11:07.943968+0000","last_fullsized":"2024-09-16T14:11:07.943968+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'645","last_scrub_stamp":"2024-09-16T14:11:07.943829+0000","last_deep_scrub":"215'645","last_deep_scrub_stamp":"2024-09-16T14:11:07.943829+0000","last_clean_scrub_stamp":"2024-09-16T14:11:07.943829+0000","objects_scrubbed":15,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:00:34.772882+0000","scrub_duration":0.026295391000000001,"objects_trimmed":0,"snaptrim_duration":0.001483112,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1924,"num_read_kb":28666,"num_write":709,"num_write_kb":21954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":17132728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"215'986","reported_seq":6393,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.735723+0000","last_change":"2024-09-16T14:11:01.735693+0000","last_active":"2024-09-16T14:11:01.735723+0000","last_peered":"2024-09-16T14:11:01.735723+0000","last_clean":"2024-09-16T14:11:01.735723+0000","last_became_active":"2024-09-16T14:07:09.544803+0000","last_became_peered":"2024-09-16T14:07:09.544803+0000","last_unstale":"2024-09-16T14:11:01.735723+0000","last_undegraded":"2024-09-16T14:11:01.735723+0000","last_fullsized":"2024-09-16T14:11:01.735723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'986","last_scrub_stamp":"2024-09-16T14:11:01.735585+0000","last_deep_scrub":"215'986","last_deep_scrub_stamp":"2024-09-16T14:11:01.735585+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.735585+0000","objects_scrubbed":18,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:55.356183+0000","scrub_duration":0.024480147000000001,"objects_trimmed":0,"snaptrim_duration":0.0050019210000000003,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9238,"num_read_kb":21812,"num_write":1162,"num_write_kb":8674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"215'1370","reported_seq":6459,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:02.748860+0000","last_change":"2024-09-16T14:11:02.748834+0000","last_active":"2024-09-16T14:11:02.748860+0000","last_peered":"2024-09-16T14:11:02.748860+0000","last_clean":"2024-09-16T14:11:02.748860+0000","last_became_active":"2024-09-16T14:10:05.123005+0000","last_became_peered":"2024-09-16T14:10:05.123005+0000","last_unstale":"2024-09-16T14:11:02.748860+0000","last_undegraded":"2024-09-16T14:11:02.748860+0000","last_fullsized":"2024-09-16T14:11:02.748860+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1370","last_scrub_stamp":"2024-09-16T14:11:02.748740+0000","last_deep_scrub":"215'1370","last_deep_scrub_stamp":"2024-09-16T14:11:02.748740+0000","last_clean_scrub_stamp":"2024-09-16T14:11:02.748740+0000","objects_scrubbed":19,"log_size":1370,"log_dups_size":0,"ondisk_log_size":1370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:08.089103+0000","scrub_duration":0.031876672000000002,"objects_trimmed":0,"snaptrim_duration":0.0039366710000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8929,"num_read_kb":33272,"num_write":1987,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":517,"num_bytes_recovered":4502377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"215'774","reported_seq":13976,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:03.970164+0000","last_change":"2024-09-16T14:11:03.970113+0000","last_active":"2024-09-16T14:11:03.970164+0000","last_peered":"2024-09-16T14:11:03.970164+0000","last_clean":"2024-09-16T14:11:03.970164+0000","last_became_active":"2024-09-16T14:10:05.377635+0000","last_became_peered":"2024-09-16T14:10:05.377635+0000","last_unstale":"2024-09-16T14:11:03.970164+0000","last_undegraded":"2024-09-16T14:11:03.970164+0000","last_fullsized":"2024-09-16T14:11:03.970164+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'774","last_scrub_stamp":"2024-09-16T14:11:03.969989+0000","last_deep_scrub":"215'774","last_deep_scrub_stamp":"2024-09-16T14:11:03.969989+0000","last_clean_scrub_stamp":"2024-09-16T14:11:03.969989+0000","objects_scrubbed":23,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:40.975725+0000","scrub_duration":0.031613308,"objects_trimmed":0,"snaptrim_duration":0.00089383699999999997,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":14525,"num_read_kb":50269,"num_write":689,"num_write_kb":12286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":11596768,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"215'601","reported_seq":5226,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:00.892229+0000","last_change":"2024-09-16T14:11:00.892194+0000","last_active":"2024-09-16T14:11:00.892229+0000","last_peered":"2024-09-16T14:11:00.892229+0000","last_clean":"2024-09-16T14:11:00.892229+0000","last_became_active":"2024-09-16T14:10:19.498121+0000","last_became_peered":"2024-09-16T14:10:19.498121+0000","last_unstale":"2024-09-16T14:11:00.892229+0000","last_undegraded":"2024-09-16T14:11:00.892229+0000","last_fullsized":"2024-09-16T14:11:00.892229+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'601","last_scrub_stamp":"2024-09-16T14:11:00.892082+0000","last_deep_scrub":"215'601","last_deep_scrub_stamp":"2024-09-16T14:11:00.892082+0000","last_clean_scrub_stamp":"2024-09-16T14:11:00.892082+0000","objects_scrubbed":25,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:30.076930+0000","scrub_duration":0.035753736000000001,"objects_trimmed":0,"snaptrim_duration":0.002716544,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4971,"num_read_kb":28564,"num_write":500,"num_write_kb":10173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":50024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"215'734","reported_seq":7846,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:35.577087+0000","last_change":"2024-09-16T14:11:35.577042+0000","last_active":"2024-09-16T14:11:35.577087+0000","last_peered":"2024-09-16T14:11:35.577087+0000","last_clean":"2024-09-16T14:11:35.577087+0000","last_became_active":"2024-09-16T14:10:06.388650+0000","last_became_peered":"2024-09-16T14:10:06.388650+0000","last_unstale":"2024-09-16T14:11:35.577087+0000","last_undegraded":"2024-09-16T14:11:35.577087+0000","last_fullsized":"2024-09-16T14:11:35.577087+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'734","last_scrub_stamp":"2024-09-16T14:11:35.576939+0000","last_deep_scrub":"215'734","last_deep_scrub_stamp":"2024-09-16T14:11:35.576939+0000","last_clean_scrub_stamp":"2024-09-16T14:11:35.576939+0000","objects_scrubbed":11,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:09.917979+0000","scrub_duration":0.021172096000000001,"objects_trimmed":0,"snaptrim_duration":0.0054744670000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9236,"num_read_kb":38811,"num_write":682,"num_write_kb":10309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":10174643,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":76,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:24.582615+0000","last_change":"2024-09-16T14:11:24.582546+0000","last_active":"2024-09-16T14:11:24.582615+0000","last_peered":"2024-09-16T14:11:24.582615+0000","last_clean":"2024-09-16T14:11:24.582615+0000","last_became_active":"2024-09-16T14:10:17.555698+0000","last_became_peered":"2024-09-16T14:10:17.555698+0000","last_unstale":"2024-09-16T14:11:24.582615+0000","last_undegraded":"2024-09-16T14:11:24.582615+0000","last_fullsized":"2024-09-16T14:11:24.582615+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:11:24.582295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:11:24.582295+0000","last_clean_scrub_stamp":"2024-09-16T14:11:24.582295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:10.110320+0000","scrub_duration":0.0032092190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"215'913","reported_seq":6204,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.946914+0000","last_change":"2024-09-16T14:10:57.946822+0000","last_active":"2024-09-16T14:10:57.946914+0000","last_peered":"2024-09-16T14:10:57.946914+0000","last_clean":"2024-09-16T14:10:57.946914+0000","last_became_active":"2024-09-16T14:10:25.385245+0000","last_became_peered":"2024-09-16T14:10:25.385245+0000","last_unstale":"2024-09-16T14:10:57.946914+0000","last_undegraded":"2024-09-16T14:10:57.946914+0000","last_fullsized":"2024-09-16T14:10:57.946914+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'913","last_scrub_stamp":"2024-09-16T14:10:57.946694+0000","last_deep_scrub":"215'913","last_deep_scrub_stamp":"2024-09-16T14:10:57.946694+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.946694+0000","objects_scrubbed":16,"log_size":913,"log_dups_size":0,"ondisk_log_size":913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:11.677456+0000","scrub_duration":0.024780823,"objects_trimmed":0,"snaptrim_duration":0.0026361219999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":5530,"num_read_kb":25811,"num_write":916,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":731,"num_bytes_recovered":15701912,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:06.947533+0000","last_change":"2024-09-16T14:11:06.947388+0000","last_active":"2024-09-16T14:11:06.947533+0000","last_peered":"2024-09-16T14:11:06.947533+0000","last_clean":"2024-09-16T14:11:06.947533+0000","last_became_active":"2024-09-16T14:10:16.471977+0000","last_became_peered":"2024-09-16T14:10:16.471977+0000","last_unstale":"2024-09-16T14:11:06.947533+0000","last_undegraded":"2024-09-16T14:11:06.947533+0000","last_fullsized":"2024-09-16T14:11:06.947533+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:11:06.947138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:11:06.947138+0000","last_clean_scrub_stamp":"2024-09-16T14:11:06.947138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:34.633141+0000","scrub_duration":0.0036792880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"215'991","reported_seq":7760,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.016287+0000","last_change":"2024-09-16T14:11:01.016189+0000","last_active":"2024-09-16T14:11:01.016287+0000","last_peered":"2024-09-16T14:11:01.016287+0000","last_clean":"2024-09-16T14:11:01.016287+0000","last_became_active":"2024-09-16T14:09:02.366959+0000","last_became_peered":"2024-09-16T14:09:02.366959+0000","last_unstale":"2024-09-16T14:11:01.016287+0000","last_undegraded":"2024-09-16T14:11:01.016287+0000","last_fullsized":"2024-09-16T14:11:01.016287+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'991","last_scrub_stamp":"2024-09-16T14:11:01.016033+0000","last_deep_scrub":"215'991","last_deep_scrub_stamp":"2024-09-16T14:11:01.016033+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.016033+0000","objects_scrubbed":24,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:02.743229+0000","scrub_duration":0.032062161999999998,"objects_trimmed":0,"snaptrim_duration":0.002259535,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":10392,"num_read_kb":45651,"num_write":1244,"num_write_kb":30465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:58.748481+0000","last_change":"2024-09-16T14:10:58.748412+0000","last_active":"2024-09-16T14:10:58.748481+0000","last_peered":"2024-09-16T14:10:58.748481+0000","last_clean":"2024-09-16T14:10:58.748481+0000","last_became_active":"2024-09-16T14:10:00.102521+0000","last_became_peered":"2024-09-16T14:10:00.102521+0000","last_unstale":"2024-09-16T14:10:58.748481+0000","last_undegraded":"2024-09-16T14:10:58.748481+0000","last_fullsized":"2024-09-16T14:10:58.748481+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:10:58.748188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:10:58.748188+0000","last_clean_scrub_stamp":"2024-09-16T14:10:58.748188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:54.126496+0000","scrub_duration":0.0037129939999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"215'647","reported_seq":3980,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:02.252304+0000","last_change":"2024-09-16T14:11:02.252277+0000","last_active":"2024-09-16T14:11:02.252304+0000","last_peered":"2024-09-16T14:11:02.252304+0000","last_clean":"2024-09-16T14:11:02.252304+0000","last_became_active":"2024-09-16T14:07:09.546620+0000","last_became_peered":"2024-09-16T14:07:09.546620+0000","last_unstale":"2024-09-16T14:11:02.252304+0000","last_undegraded":"2024-09-16T14:11:02.252304+0000","last_fullsized":"2024-09-16T14:11:02.252304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'647","last_scrub_stamp":"2024-09-16T14:11:02.252154+0000","last_deep_scrub":"215'647","last_deep_scrub_stamp":"2024-09-16T14:11:02.252154+0000","last_clean_scrub_stamp":"2024-09-16T14:11:02.252154+0000","objects_scrubbed":17,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:29.668999+0000","scrub_duration":0.026147526000000001,"objects_trimmed":0,"snaptrim_duration":0.001012095,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4560,"num_read_kb":16131,"num_write":597,"num_write_kb":10119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":218,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:59.755353+0000","last_change":"2024-09-16T14:10:59.755275+0000","last_active":"2024-09-16T14:10:59.755353+0000","last_peered":"2024-09-16T14:10:59.755353+0000","last_clean":"2024-09-16T14:10:59.755353+0000","last_became_active":"2024-09-16T14:07:00.571030+0000","last_became_peered":"2024-09-16T14:07:00.571030+0000","last_unstale":"2024-09-16T14:10:59.755353+0000","last_undegraded":"2024-09-16T14:10:59.755353+0000","last_fullsized":"2024-09-16T14:10:59.755353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:10:59.755046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:10:59.755046+0000","last_clean_scrub_stamp":"2024-09-16T14:10:59.755046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:47:48.201040+0000","scrub_duration":0.003186361,"objects_trimmed":0,"snaptrim_duration":0.00071913000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"215'792","reported_seq":9836,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.981555+0000","last_change":"2024-09-16T14:11:04.981524+0000","last_active":"2024-09-16T14:11:04.981555+0000","last_peered":"2024-09-16T14:11:04.981555+0000","last_clean":"2024-09-16T14:11:04.981555+0000","last_became_active":"2024-09-16T14:07:09.546543+0000","last_became_peered":"2024-09-16T14:07:09.546543+0000","last_unstale":"2024-09-16T14:11:04.981555+0000","last_undegraded":"2024-09-16T14:11:04.981555+0000","last_fullsized":"2024-09-16T14:11:04.981555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'792","last_scrub_stamp":"2024-09-16T14:11:04.981436+0000","last_deep_scrub":"215'792","last_deep_scrub_stamp":"2024-09-16T14:11:04.981436+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.981436+0000","objects_scrubbed":12,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:40.393594+0000","scrub_duration":0.019826035999999998,"objects_trimmed":0,"snaptrim_duration":0.001064281,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":11041,"num_read_kb":12619,"num_write":857,"num_write_kb":5348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"215'1434","reported_seq":5266,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.815104+0000","last_change":"2024-09-16T14:10:57.815074+0000","last_active":"2024-09-16T14:10:57.815104+0000","last_peered":"2024-09-16T14:10:57.815104+0000","last_clean":"2024-09-16T14:10:57.815104+0000","last_became_active":"2024-09-16T14:10:19.505692+0000","last_became_peered":"2024-09-16T14:10:19.505692+0000","last_unstale":"2024-09-16T14:10:57.815104+0000","last_undegraded":"2024-09-16T14:10:57.815104+0000","last_fullsized":"2024-09-16T14:10:57.815104+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1434","last_scrub_stamp":"2024-09-16T14:10:57.814968+0000","last_deep_scrub":"215'1434","last_deep_scrub_stamp":"2024-09-16T14:10:57.814968+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.814968+0000","objects_scrubbed":23,"log_size":1434,"log_dups_size":0,"ondisk_log_size":1434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:44:00.620195+0000","scrub_duration":0.032517719,"objects_trimmed":0,"snaptrim_duration":0.0040141910000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":9687,"num_read_kb":19616,"num_write":1641,"num_write_kb":10135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"215'3","reported_seq":68,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:07.593489+0000","last_change":"2024-09-16T14:11:07.593404+0000","last_active":"2024-09-16T14:11:07.593489+0000","last_peered":"2024-09-16T14:11:07.593489+0000","last_clean":"2024-09-16T14:11:07.593489+0000","last_became_active":"2024-09-16T14:10:00.104539+0000","last_became_peered":"2024-09-16T14:10:00.104539+0000","last_unstale":"2024-09-16T14:11:07.593489+0000","last_undegraded":"2024-09-16T14:11:07.593489+0000","last_fullsized":"2024-09-16T14:11:07.593489+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'3","last_scrub_stamp":"2024-09-16T14:11:07.593178+0000","last_deep_scrub":"215'3","last_deep_scrub_stamp":"2024-09-16T14:11:07.593178+0000","last_clean_scrub_stamp":"2024-09-16T14:11:07.593178+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:05.249880+0000","scrub_duration":0.0089886010000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"215'1337","reported_seq":6578,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:05.735842+0000","last_change":"2024-09-16T14:11:05.735811+0000","last_active":"2024-09-16T14:11:05.735842+0000","last_peered":"2024-09-16T14:11:05.735842+0000","last_clean":"2024-09-16T14:11:05.735842+0000","last_became_active":"2024-09-16T14:10:26.385527+0000","last_became_peered":"2024-09-16T14:10:26.385527+0000","last_unstale":"2024-09-16T14:11:05.735842+0000","last_undegraded":"2024-09-16T14:11:05.735842+0000","last_fullsized":"2024-09-16T14:11:05.735842+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1337","last_scrub_stamp":"2024-09-16T14:11:05.735690+0000","last_deep_scrub":"215'1337","last_deep_scrub_stamp":"2024-09-16T14:11:05.735690+0000","last_clean_scrub_stamp":"2024-09-16T14:11:05.735690+0000","objects_scrubbed":21,"log_size":1337,"log_dups_size":0,"ondisk_log_size":1337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:31.726396+0000","scrub_duration":0.032108896999999997,"objects_trimmed":0,"snaptrim_duration":0.0051618139999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":9488,"num_read_kb":24272,"num_write":1738,"num_write_kb":10082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":8428888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.310710+0000","last_change":"2024-09-16T14:11:01.310645+0000","last_active":"2024-09-16T14:11:01.310710+0000","last_peered":"2024-09-16T14:11:01.310710+0000","last_clean":"2024-09-16T14:11:01.310710+0000","last_became_active":"2024-09-16T14:10:21.387173+0000","last_became_peered":"2024-09-16T14:10:21.387173+0000","last_unstale":"2024-09-16T14:11:01.310710+0000","last_undegraded":"2024-09-16T14:11:01.310710+0000","last_fullsized":"2024-09-16T14:11:01.310710+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:11:01.310374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:11:01.310374+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.310374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:06.388664+0000","scrub_duration":0.003307089,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"215'843","reported_seq":7064,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.281157+0000","last_change":"2024-09-16T14:10:57.281110+0000","last_active":"2024-09-16T14:10:57.281157+0000","last_peered":"2024-09-16T14:10:57.281157+0000","last_clean":"2024-09-16T14:10:57.281157+0000","last_became_active":"2024-09-16T14:07:09.546466+0000","last_became_peered":"2024-09-16T14:07:09.546466+0000","last_unstale":"2024-09-16T14:10:57.281157+0000","last_undegraded":"2024-09-16T14:10:57.281157+0000","last_fullsized":"2024-09-16T14:10:57.281157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'843","last_scrub_stamp":"2024-09-16T14:10:57.281004+0000","last_deep_scrub":"215'843","last_deep_scrub_stamp":"2024-09-16T14:10:57.281004+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.281004+0000","objects_scrubbed":23,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:32:12.179239+0000","scrub_duration":0.032072307000000001,"objects_trimmed":0,"snaptrim_duration":0.001233734,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":8272,"num_read_kb":21183,"num_write":947,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:19.578577+0000","last_change":"2024-09-16T14:11:19.578521+0000","last_active":"2024-09-16T14:11:19.578577+0000","last_peered":"2024-09-16T14:11:19.578577+0000","last_clean":"2024-09-16T14:11:19.578577+0000","last_became_active":"2024-09-16T14:10:00.104003+0000","last_became_peered":"2024-09-16T14:10:00.104003+0000","last_unstale":"2024-09-16T14:11:19.578577+0000","last_undegraded":"2024-09-16T14:11:19.578577+0000","last_fullsized":"2024-09-16T14:11:19.578577+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:11:19.578246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:11:19.578246+0000","last_clean_scrub_stamp":"2024-09-16T14:11:19.578246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:24:39.413538+0000","scrub_duration":0.0036936320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"215'681","reported_seq":15134,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:12.881710+0000","last_change":"2024-09-16T14:11:12.881592+0000","last_active":"2024-09-16T14:11:12.881710+0000","last_peered":"2024-09-16T14:11:12.881710+0000","last_clean":"2024-09-16T14:11:12.881710+0000","last_became_active":"2024-09-16T14:09:02.369873+0000","last_became_peered":"2024-09-16T14:09:02.369873+0000","last_unstale":"2024-09-16T14:11:12.881710+0000","last_undegraded":"2024-09-16T14:11:12.881710+0000","last_fullsized":"2024-09-16T14:11:12.881710+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'681","last_scrub_stamp":"2024-09-16T14:11:12.881388+0000","last_deep_scrub":"215'681","last_deep_scrub_stamp":"2024-09-16T14:11:12.881388+0000","last_clean_scrub_stamp":"2024-09-16T14:11:12.881388+0000","objects_scrubbed":14,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:47.689768+0000","scrub_duration":0.021531431,"objects_trimmed":0,"snaptrim_duration":0.0020715640000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":14807,"num_read_kb":52699,"num_write":621,"num_write_kb":10498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"215'2","reported_seq":68,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:57.743051+0000","last_change":"2024-09-16T14:10:57.743012+0000","last_active":"2024-09-16T14:10:57.743051+0000","last_peered":"2024-09-16T14:10:57.743051+0000","last_clean":"2024-09-16T14:10:57.743051+0000","last_became_active":"2024-09-16T14:10:00.105302+0000","last_became_peered":"2024-09-16T14:10:00.105302+0000","last_unstale":"2024-09-16T14:10:57.743051+0000","last_undegraded":"2024-09-16T14:10:57.743051+0000","last_fullsized":"2024-09-16T14:10:57.743051+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'2","last_scrub_stamp":"2024-09-16T14:10:57.742863+0000","last_deep_scrub":"215'2","last_deep_scrub_stamp":"2024-09-16T14:10:57.742863+0000","last_clean_scrub_stamp":"2024-09-16T14:10:57.742863+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:11.383869+0000","scrub_duration":0.0043886649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"215'74","reported_seq":116,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.441740+0000","last_change":"2024-09-16T14:11:01.441669+0000","last_active":"2024-09-16T14:11:01.441740+0000","last_peered":"2024-09-16T14:11:01.441740+0000","last_clean":"2024-09-16T14:11:01.441740+0000","last_became_active":"2024-09-16T14:09:14.385418+0000","last_became_peered":"2024-09-16T14:09:14.385418+0000","last_unstale":"2024-09-16T14:11:01.441740+0000","last_undegraded":"2024-09-16T14:11:01.441740+0000","last_fullsized":"2024-09-16T14:11:01.441740+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'74","last_scrub_stamp":"2024-09-16T14:11:01.441487+0000","last_deep_scrub":"215'74","last_deep_scrub_stamp":"2024-09-16T14:11:01.441487+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.441487+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:52:47.519144+0000","scrub_duration":0.010507404999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"215'727","reported_seq":4321,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:59.994214+0000","last_change":"2024-09-16T14:10:59.994163+0000","last_active":"2024-09-16T14:10:59.994214+0000","last_peered":"2024-09-16T14:10:59.994214+0000","last_clean":"2024-09-16T14:10:59.994214+0000","last_became_active":"2024-09-16T14:10:26.384950+0000","last_became_peered":"2024-09-16T14:10:26.384950+0000","last_unstale":"2024-09-16T14:10:59.994214+0000","last_undegraded":"2024-09-16T14:10:59.994214+0000","last_fullsized":"2024-09-16T14:10:59.994214+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'727","last_scrub_stamp":"2024-09-16T14:10:59.994060+0000","last_deep_scrub":"215'727","last_deep_scrub_stamp":"2024-09-16T14:10:59.994060+0000","last_clean_scrub_stamp":"2024-09-16T14:10:59.994060+0000","objects_scrubbed":23,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:43.925119+0000","scrub_duration":0.028713651999999999,"objects_trimmed":0,"snaptrim_duration":0.00088242400000000003,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4245,"num_read_kb":11739,"num_write":722,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":2087145,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"215'642","reported_seq":6992,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.721211+0000","last_change":"2024-09-16T14:11:04.721181+0000","last_active":"2024-09-16T14:11:04.721211+0000","last_peered":"2024-09-16T14:11:04.721211+0000","last_clean":"2024-09-16T14:11:04.721211+0000","last_became_active":"2024-09-16T14:10:08.396086+0000","last_became_peered":"2024-09-16T14:10:08.396086+0000","last_unstale":"2024-09-16T14:11:04.721211+0000","last_undegraded":"2024-09-16T14:11:04.721211+0000","last_fullsized":"2024-09-16T14:11:04.721211+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'642","last_scrub_stamp":"2024-09-16T14:11:04.721065+0000","last_deep_scrub":"215'642","last_deep_scrub_stamp":"2024-09-16T14:11:04.721065+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.721065+0000","objects_scrubbed":19,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:39.505932+0000","scrub_duration":0.023087542999999999,"objects_trimmed":0,"snaptrim_duration":0.00321149,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6657,"num_read_kb":16604,"num_write":570,"num_write_kb":11897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":10317335,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"215'871","reported_seq":6931,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:05.691498+0000","last_change":"2024-09-16T14:11:05.691461+0000","last_active":"2024-09-16T14:11:05.691498+0000","last_peered":"2024-09-16T14:11:05.691498+0000","last_clean":"2024-09-16T14:11:05.691498+0000","last_became_active":"2024-09-16T14:10:07.390964+0000","last_became_peered":"2024-09-16T14:10:07.390964+0000","last_unstale":"2024-09-16T14:11:05.691498+0000","last_undegraded":"2024-09-16T14:11:05.691498+0000","last_fullsized":"2024-09-16T14:11:05.691498+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'871","last_scrub_stamp":"2024-09-16T14:11:05.691316+0000","last_deep_scrub":"215'871","last_deep_scrub_stamp":"2024-09-16T14:11:05.691316+0000","last_clean_scrub_stamp":"2024-09-16T14:11:05.691316+0000","objects_scrubbed":21,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:09.065273+0000","scrub_duration":0.029089304,"objects_trimmed":0,"snaptrim_duration":0.0041712620000000002,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":7231,"num_read_kb":39110,"num_write":775,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":49748621,"num_keys_recovered":196,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"215'770","reported_seq":5216,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:04.334984+0000","last_change":"2024-09-16T14:11:04.334949+0000","last_active":"2024-09-16T14:11:04.334984+0000","last_peered":"2024-09-16T14:11:04.334984+0000","last_clean":"2024-09-16T14:11:04.334984+0000","last_became_active":"2024-09-16T14:10:24.410718+0000","last_became_peered":"2024-09-16T14:10:24.410718+0000","last_unstale":"2024-09-16T14:11:04.334984+0000","last_undegraded":"2024-09-16T14:11:04.334984+0000","last_fullsized":"2024-09-16T14:11:04.334984+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'770","last_scrub_stamp":"2024-09-16T14:11:04.334853+0000","last_deep_scrub":"215'770","last_deep_scrub_stamp":"2024-09-16T14:11:04.334853+0000","last_clean_scrub_stamp":"2024-09-16T14:11:04.334853+0000","objects_scrubbed":23,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:11:33.852245+0000","scrub_duration":0.032357058000000001,"objects_trimmed":0,"snaptrim_duration":0.00064224900000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":7094,"num_read_kb":44664,"num_write":801,"num_write_kb":22800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":19444502,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"215'645","reported_seq":8169,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:59.892384+0000","last_change":"2024-09-16T14:10:59.892354+0000","last_active":"2024-09-16T14:10:59.892384+0000","last_peered":"2024-09-16T14:10:59.892384+0000","last_clean":"2024-09-16T14:10:59.892384+0000","last_became_active":"2024-09-16T14:10:25.384634+0000","last_became_peered":"2024-09-16T14:10:25.384634+0000","last_unstale":"2024-09-16T14:10:59.892384+0000","last_undegraded":"2024-09-16T14:10:59.892384+0000","last_fullsized":"2024-09-16T14:10:59.892384+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'645","last_scrub_stamp":"2024-09-16T14:10:59.892274+0000","last_deep_scrub":"215'645","last_deep_scrub_stamp":"2024-09-16T14:10:59.892274+0000","last_clean_scrub_stamp":"2024-09-16T14:10:59.892274+0000","objects_scrubbed":24,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:07.283676+0000","scrub_duration":0.029250721,"objects_trimmed":0,"snaptrim_duration":0.0034031370000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7377,"num_read_kb":27693,"num_write":494,"num_write_kb":13033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":23836787,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"215'506","reported_seq":2815,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:17.609692+0000","last_change":"2024-09-16T14:11:17.609657+0000","last_active":"2024-09-16T14:11:17.609692+0000","last_peered":"2024-09-16T14:11:17.609692+0000","last_clean":"2024-09-16T14:11:17.609692+0000","last_became_active":"2024-09-16T14:10:04.123286+0000","last_became_peered":"2024-09-16T14:10:04.123286+0000","last_unstale":"2024-09-16T14:11:17.609692+0000","last_undegraded":"2024-09-16T14:11:17.609692+0000","last_fullsized":"2024-09-16T14:11:17.609692+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'506","last_scrub_stamp":"2024-09-16T14:11:17.609527+0000","last_deep_scrub":"215'506","last_deep_scrub_stamp":"2024-09-16T14:11:17.609527+0000","last_clean_scrub_stamp":"2024-09-16T14:11:17.609527+0000","objects_scrubbed":10,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:31:48.373409+0000","scrub_duration":0.018496696,"objects_trimmed":0,"snaptrim_duration":0.0054803270000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":25169,"num_write":396,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":20371845,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"215'524","reported_seq":7424,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:02.941800+0000","last_change":"2024-09-16T14:11:02.941748+0000","last_active":"2024-09-16T14:11:02.941800+0000","last_peered":"2024-09-16T14:11:02.941800+0000","last_clean":"2024-09-16T14:11:02.941800+0000","last_became_active":"2024-09-16T14:10:16.487713+0000","last_became_peered":"2024-09-16T14:10:16.487713+0000","last_unstale":"2024-09-16T14:11:02.941800+0000","last_undegraded":"2024-09-16T14:11:02.941800+0000","last_fullsized":"2024-09-16T14:11:02.941800+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'524","last_scrub_stamp":"2024-09-16T14:11:02.941570+0000","last_deep_scrub":"215'524","last_deep_scrub_stamp":"2024-09-16T14:11:02.941570+0000","last_clean_scrub_stamp":"2024-09-16T14:11:02.941570+0000","objects_scrubbed":14,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:12.209788+0000","scrub_duration":0.020611933999999998,"objects_trimmed":0,"snaptrim_duration":0.00072767700000000003,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7823,"num_read_kb":31641,"num_write":482,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"215'1017","reported_seq":6924,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:01.919365+0000","last_change":"2024-09-16T14:11:01.919302+0000","last_active":"2024-09-16T14:11:01.919365+0000","last_peered":"2024-09-16T14:11:01.919365+0000","last_clean":"2024-09-16T14:11:01.919365+0000","last_became_active":"2024-09-16T14:10:19.501036+0000","last_became_peered":"2024-09-16T14:10:19.501036+0000","last_unstale":"2024-09-16T14:11:01.919365+0000","last_undegraded":"2024-09-16T14:11:01.919365+0000","last_fullsized":"2024-09-16T14:11:01.919365+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1017","last_scrub_stamp":"2024-09-16T14:11:01.919152+0000","last_deep_scrub":"215'1017","last_deep_scrub_stamp":"2024-09-16T14:11:01.919152+0000","last_clean_scrub_stamp":"2024-09-16T14:11:01.919152+0000","objects_scrubbed":14,"log_size":1017,"log_dups_size":0,"ondisk_log_size":1017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:25.383116+0000","scrub_duration":0.025920904000000002,"objects_trimmed":0,"snaptrim_duration":0.0041967539999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":10106,"num_read_kb":24848,"num_write":1204,"num_write_kb":15655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":8219627,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"215'638","reported_seq":12664,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:10:59.401294+0000","last_change":"2024-09-16T14:10:59.401266+0000","last_active":"2024-09-16T14:10:59.401294+0000","last_peered":"2024-09-16T14:10:59.401294+0000","last_clean":"2024-09-16T14:10:59.401294+0000","last_became_active":"2024-09-16T14:09:18.694205+0000","last_became_peered":"2024-09-16T14:09:18.694205+0000","last_unstale":"2024-09-16T14:10:59.401294+0000","last_undegraded":"2024-09-16T14:10:59.401294+0000","last_fullsized":"2024-09-16T14:10:59.401294+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'638","last_scrub_stamp":"2024-09-16T14:10:59.401187+0000","last_deep_scrub":"215'638","last_deep_scrub_stamp":"2024-09-16T14:10:59.401187+0000","last_clean_scrub_stamp":"2024-09-16T14:10:59.401187+0000","objects_scrubbed":13,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:46.492819+0000","scrub_duration":0.021501461999999999,"objects_trimmed":0,"snaptrim_duration":0.001114802,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13307,"num_read_kb":24046,"num_write":748,"num_write_kb":16341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":11353438,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"215'1001","reported_seq":6269,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:09.614897+0000","last_change":"2024-09-16T14:11:09.614839+0000","last_active":"2024-09-16T14:11:09.614897+0000","last_peered":"2024-09-16T14:11:09.614897+0000","last_clean":"2024-09-16T14:11:09.614897+0000","last_became_active":"2024-09-16T14:10:08.399369+0000","last_became_peered":"2024-09-16T14:10:08.399369+0000","last_unstale":"2024-09-16T14:11:09.614897+0000","last_undegraded":"2024-09-16T14:11:09.614897+0000","last_fullsized":"2024-09-16T14:11:09.614897+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'1001","last_scrub_stamp":"2024-09-16T14:11:09.614716+0000","last_deep_scrub":"215'1001","last_deep_scrub_stamp":"2024-09-16T14:11:09.614716+0000","last_clean_scrub_stamp":"2024-09-16T14:11:09.614716+0000","objects_scrubbed":15,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:24:48.420756+0000","scrub_duration":0.022699634999999999,"objects_trimmed":0,"snaptrim_duration":0.0036375169999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6839,"num_read_kb":16656,"num_write":1300,"num_write_kb":11208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":7816767,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"215'768","reported_seq":6864,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:06.209704+0000","last_change":"2024-09-16T14:11:06.209670+0000","last_active":"2024-09-16T14:11:06.209704+0000","last_peered":"2024-09-16T14:11:06.209704+0000","last_clean":"2024-09-16T14:11:06.209704+0000","last_became_active":"2024-09-16T14:10:05.121914+0000","last_became_peered":"2024-09-16T14:10:05.121914+0000","last_unstale":"2024-09-16T14:11:06.209704+0000","last_undegraded":"2024-09-16T14:11:06.209704+0000","last_fullsized":"2024-09-16T14:11:06.209704+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'768","last_scrub_stamp":"2024-09-16T14:11:06.209574+0000","last_deep_scrub":"215'768","last_deep_scrub_stamp":"2024-09-16T14:11:06.209574+0000","last_clean_scrub_stamp":"2024-09-16T14:11:06.209574+0000","objects_scrubbed":27,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:38.550773+0000","scrub_duration":0.042191224999999999,"objects_trimmed":0,"snaptrim_duration":0.0025226160000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":7908,"num_read_kb":56867,"num_write":705,"num_write_kb":31085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":28924045,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"215'763","reported_seq":7866,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-16T14:11:33.613680+0000","last_change":"2024-09-16T14:11:33.613640+0000","last_active":"2024-09-16T14:11:33.613680+0000","last_peered":"2024-09-16T14:11:33.613680+0000","last_clean":"2024-09-16T14:11:33.613680+0000","last_became_active":"2024-09-16T14:10:21.385616+0000","last_became_peered":"2024-09-16T14:10:21.385616+0000","last_unstale":"2024-09-16T14:11:33.613680+0000","last_undegraded":"2024-09-16T14:11:33.613680+0000","last_fullsized":"2024-09-16T14:11:33.613680+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'763","last_scrub_stamp":"2024-09-16T14:11:33.613453+0000","last_deep_scrub":"215'763","last_deep_scrub_stamp":"2024-09-16T14:11:33.613453+0000","last_clean_scrub_stamp":"2024-09-16T14:11:33.613453+0000","objects_scrubbed":10,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:08.505015+0000","scrub_duration":0.018000639999999998,"objects_trimmed":0,"snaptrim_duration":0.0047871169999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8482,"num_read_kb":31842,"num_write":873,"num_write_kb":18389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":41601355,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":588,"num_read":281369,"num_read_kb":918178,"num_write":31085,"num_write_kb":441735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9458,"num_bytes_recovered":336276108,"num_keys_recovered":883,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37982,"internal_metadata":0},"log_size":27681,"ondisk_log_size":27681,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":17805,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509502,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304044,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067796,"statfs":{"total":96636764160,"available":96325423104,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:11:17 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.099},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.99399999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83599999999999997,"15min":0.85499999999999998},"min":{"1min":0.51200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2210000000000001,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.84299999999999997,"15min":0.84199999999999997},"min":{"1min":0.57399999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.24,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.77900000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.258,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.55500000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1839999999999999,"5min":5.0810000000000004,"15min":5.0810000000000004},"last":1.125}]},{"osd":3,"last update":"Mon Sep 16 14:10:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.0149999999999999,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.2689999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.244},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.0089999999999999},"min":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.829,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":0.84399999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.94299999999999995,"15min":0.94799999999999995},"min":{"1min":0.60499999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.3080000000000001,"5min":3.246,"15min":3.246},"last":1.165},{"interface":"front","average":{"1min":1.083,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.60699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.526,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.876}]},{"osd":5,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.012,"15min":1.02},"min":{"1min":0.63300000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.351,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.0669999999999999,"15min":1.0289999999999999},"min":{"1min":0.505,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5840000000000001,"5min":3.3919999999999999,"15min":3.3919999999999999},"last":1.0309999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.107,"15min":1.077},"min":{"1min":0.749,"5min":0.439,"15min":0.439},"max":{"1min":1.6990000000000001,"5min":3.492,"15min":3.492},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.224,"5min":1.141,"15min":1.137},"min":{"1min":0.80600000000000005,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8169999999999999,"5min":2.8679999999999999,"15min":2.8679999999999999},"last":1.194}]}]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312368,"kb_used_data":2536,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059472,"statfs":{"total":96636764160,"available":96316899328,"internally_reserved":0,"allocated":2596864,"data_stored":1250825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:11:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.95099999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.84899999999999998,"15min":0.91100000000000003},"min":{"1min":0.49099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3109999999999999,"5min":1.413,"15min":1.413},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.92300000000000004,"15min":0.98399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.419,"5min":2.6920000000000002,"15min":2.6920000000000002},"last":1.1359999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.84199999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":2.778,"15min":2.778},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82099999999999995,"15min":0.86899999999999999},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.236,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.0129999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:11:33 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.93799999999999994,"15min":0.94399999999999995},"min":{"1min":0.58199999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.4330000000000001,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.042},{"interface":"front","average":{"1min":1.1220000000000001,"5min":0.93000000000000005,"15min":0.89600000000000002},"min":{"1min":0.78200000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.359,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.2070000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86299999999999999,"15min":0.94699999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.272,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.97099999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.3839999999999999,"5min":3.3719999999999999,"15min":3.3719999999999999},"last":0.74399999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94799999999999995,"15min":0.96599999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1930000000000001,"5min":4.976,"15min":4.976},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.98599999999999999,"15min":1.04},"min":{"1min":0.42799999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.113,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.80200000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:11:02 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0409999999999999,"15min":1.0609999999999999},"min":{"1min":0.54400000000000004,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4770000000000001,"5min":3.4319999999999999,"15min":3.4319999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0649999999999999,"15min":1.1299999999999999},"min":{"1min":0.58699999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3109999999999999,"5min":2.8799999999999999,"15min":2.8799999999999999},"last":1.0720000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509502,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308264,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063576,"statfs":{"total":96636764160,"available":96321101824,"internally_reserved":0,"allocated":2654208,"data_stored":1841193,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.1060000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84199999999999997,"15min":0.88100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1970000000000001,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.88700000000000001,"15min":0.94999999999999996},"min":{"1min":0.36399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.258,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93000000000000005,"15min":0.91900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.143,"5min":10.999000000000001,"15min":10.999000000000001},"last":1.234},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.0249999999999999,"15min":1.0429999999999999},"min":{"1min":0.504,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.488,"5min":11.018000000000001,"15min":11.018000000000001},"last":1.1479999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:10:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82699999999999996,"15min":0.82599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1160000000000001,"5min":1.212,"15min":1.212},"last":0.79000000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.87,"15min":0.93000000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.111,"5min":1.601,"15min":1.601},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.96899999999999997,"15min":0.997},"min":{"1min":0.501,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4199999999999999,"5min":1.764,"15min":1.764},"last":1.2789999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":0.92200000000000004,"15min":0.94399999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.024,"5min":0.98599999999999999,"15min":1.054},"min":{"1min":0.47299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":1.3260000000000001,"5min":1.573,"15min":1.573},"last":0.749}]},{"osd":7,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.0109999999999999,"15min":1.0469999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.0760000000000001,"15min":1.125},"min":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.89400000000000002}]}]},{"osd":3,"up_from":83,"seq":356482285609,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309936,"kb_used_data":2536,"kb_used_omap":31,"kb_used_meta":307360,"kb_avail":94061904,"statfs":{"total":96636764160,"available":96319389696,"internally_reserved":0,"allocated":2596864,"data_stored":1250825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31864,"internal_metadata":314737544},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:11:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73899999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63100000000000001,"15min":0.628},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.3049999999999999,"5min":1.468,"15min":1.468},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67600000000000005,"15min":0.66100000000000003},"min":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0509999999999999,"5min":1.258,"15min":1.258},"last":0.82899999999999996}]},{"osd":2,"last update":"Mon Sep 16 14:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72599999999999998,"15min":0.752},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0980000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0189999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.80200000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88400000000000001,"15min":0.89800000000000002},"min":{"1min":0.66000000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.109,"5min":1.347,"15min":1.347},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.97299999999999998,"15min":0.96899999999999997},"min":{"1min":0.70299999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2689999999999999,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.97499999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:11:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.60199999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":0.98399999999999999,"15min":0.96099999999999997},"min":{"1min":0.80600000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.248,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.097}]},{"osd":6,"last update":"Mon Sep 16 14:11:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0449999999999999,"15min":1.036},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3180000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.149},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.054,"15min":1.044},"min":{"1min":0.78400000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2470000000000001,"5min":1.486,"15min":1.486},"last":0.94899999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:11:34 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.0469999999999999,"15min":1.032},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3979999999999999,"5min":1.419,"15min":1.419},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.044,"15min":1.028},"min":{"1min":0.745,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3959999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.85999999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509503,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308204,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063636,"statfs":{"total":96636764160,"available":96321163264,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.0049999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.69599999999999995,"15min":0.71499999999999997},"min":{"1min":0.41999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74299999999999999,"15min":0.76100000000000001},"min":{"1min":0.375,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.69399999999999995}]},{"osd":3,"last update":"Mon Sep 16 14:10:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.5,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.68600000000000005,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.62}]},{"osd":4,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.89100000000000001,"15min":0.877},"min":{"1min":0.77400000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.871,"15min":0.85399999999999998},"min":{"1min":0.59299999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.88600000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":0.90400000000000003,"15min":0.86799999999999999},"min":{"1min":0.63200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.97799999999999998,"15min":0.97599999999999998},"min":{"1min":0.68700000000000006,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.085}]},{"osd":6,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.68200000000000005,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.49,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.67800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.159}]},{"osd":7,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.99399999999999999,"15min":1.0009999999999999},"min":{"1min":0.64900000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.0129999999999999,"15min":0.97399999999999998},"min":{"1min":0.78700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.1140000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312408,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059432,"statfs":{"total":96636764160,"available":96316858368,"internally_reserved":0,"allocated":2637824,"data_stored":1841193,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.876},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.83799999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.74099999999999999},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.6230000000000002,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63500000000000001,"15min":0.66900000000000004},"min":{"1min":0.35799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1339999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:11:28 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78600000000000003,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2450000000000001,"5min":7.9950000000000001,"15min":7.9950000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.65400000000000003,"15min":0.64400000000000002},"min":{"1min":0.41699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.143,"5min":1.244,"15min":1.244},"last":0.76500000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84199999999999997,"15min":0.88800000000000001},"min":{"1min":0.47099999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.103,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.028},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.89900000000000002,"15min":0.92700000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0489999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.0329999999999999,"15min":1.1020000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.272,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.028,"5min":0.99199999999999999,"15min":1.0620000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.1279999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":1.032,"15min":1.0820000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.105,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.0349999999999999,"15min":1.093},"min":{"1min":0.53600000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":1.1850000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.056},"min":{"1min":0.56999999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.403,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.1200000000000001,"15min":1.1399999999999999},"min":{"1min":0.67900000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.512,"5min":7.7190000000000003,"15min":7.7190000000000003},"last":1.0780000000000001}]}]},{"osd":0,"up_from":184,"seq":790273982483,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311852,"kb_used_data":2532,"kb_used_omap":125,"kb_used_meta":309186,"kb_avail":94059988,"statfs":{"total":96636764160,"available":96317427712,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":128171,"internal_metadata":316607317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.64300000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":1.03}]},{"osd":3,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.82599999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":1.48}]},{"osd":5,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.554}]},{"osd":6,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.1559999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.262},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.1870000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542206,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299884,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071956,"statfs":{"total":96636764160,"available":96329682944,"internally_reserved":0,"allocated":2592768,"data_stored":1250806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:11:16 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.071},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.1299999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.208,"15min":1.032},"min":{"1min":0.50800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.329,"5min":37.805,"15min":37.805},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.177,"15min":1.0069999999999999},"min":{"1min":0.58899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.149,"5min":37.857999999999997,"15min":37.857999999999997},"last":0.97399999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.85399999999999998,"15min":0.86299999999999999},"min":{"1min":0.55100000000000005,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.3460000000000001,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57999999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.409,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Mon Sep 16 14:10:34 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80900000000000005,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0149999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.68000000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.1930000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.92600000000000005,"15min":0.94399999999999995},"min":{"1min":0.38200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.262,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92100000000000004,"15min":0.94099999999999995},"min":{"1min":0.44600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.385,"5min":1.611,"15min":1.611},"last":1.044}]},{"osd":6,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.95499999999999996,"15min":0.97199999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.224,"5min":1.429,"15min":1.429},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0269999999999999,"15min":1.034},"min":{"1min":0.42499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.601,"15min":1.601},"last":1.1579999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.0289999999999999,"15min":1.042},"min":{"1min":0.68000000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3480000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.123,"5min":1.081,"15min":1.0980000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.464,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.294}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6207,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5391,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29034,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8948,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:11:38.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T14:11:38.998 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T14:11:38.998 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T14:11:38.998 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T14:11:38.998 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.076 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T14:11:39.076 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T14:11:39.077 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.149 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T14:11:39.149 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T14:11:39.149 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.222 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T14:11:39.222 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T14:11:39.222 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.283 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T14:11:39.283 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T14:11:39.283 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.351 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T14:11:39.351 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T14:11:39.351 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.426 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T14:11:39.427 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T14:11:39.427 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.485 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T14:11:39.486 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T14:11:39.486 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.580 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T14:11:39.580 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T14:11:39.580 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T14:11:39.580 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.608 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T14:11:39.608 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T14:11:39.608 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.638 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T14:11:39.638 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T14:11:39.638 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T14:11:39.638 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.657 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T14:11:39.658 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T14:11:39.658 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.677 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T14:11:39.678 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T14:11:39.678 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:11:39.698 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T14:11:39.698 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T14:11:39.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-16T14:11:39.744 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:11:39.744 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T14:11:40.009 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:11:40.009 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T14:11:40.263 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:11:40.263 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T14:11:40.449 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:11:40.449 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T14:11:40.587 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:11:40.587 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T14:11:40.825 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:11:40.825 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T14:11:41.063 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:11:41.063 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T14:11:41.267 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:11:41.267 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T14:11:41.426 INFO:tasks.ceph:Archiving mon data... 2024-09-16T14:11:41.427 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/data/mon.a.tgz 2024-09-16T14:11:41.427 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-16T14:11:41.434 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.hnRJDsHDq6 2024-09-16T14:11:41.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.hnRJDsHDq6 2024-09-16T14:11:41.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.hnRJDsHDq6 2024-09-16T14:11:41.571 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.hnRJDsHDq6 is 455KB 2024-09-16T14:11:41.636 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.hnRJDsHDq6 2024-09-16T14:11:41.642 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/data/mon.c.tgz 2024-09-16T14:11:41.642 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-09-16T14:11:41.649 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.yHjeOMmBZl 2024-09-16T14:11:41.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.yHjeOMmBZl 2024-09-16T14:11:41.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.yHjeOMmBZl 2024-09-16T14:11:41.857 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.yHjeOMmBZl is 615KB 2024-09-16T14:11:41.885 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.yHjeOMmBZl 2024-09-16T14:11:41.891 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/data/mon.b.tgz 2024-09-16T14:11:41.891 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-16T14:11:41.899 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.Exafvhebly 2024-09-16T14:11:41.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Exafvhebly 2024-09-16T14:11:42.100 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.Exafvhebly 2024-09-16T14:11:42.120 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.Exafvhebly is 615KB 2024-09-16T14:11:42.185 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.Exafvhebly 2024-09-16T14:11:42.192 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T14:11:42.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T14:11:42.194 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T14:11:42.210 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T14:11:42.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T14:11:42.212 DEBUG:teuthology.orchestra.run.smithi164:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T14:11:42.236 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T14:11:42.278 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T14:11:42.284 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T14:11:42.286 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/remote/smithi079/crash 2024-09-16T14:11:42.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T14:11:42.307 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/remote/smithi164/crash 2024-09-16T14:11:42.308 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T14:11:42.327 INFO:tasks.ceph:Compressing logs... 2024-09-16T14:11:42.327 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T14:11:42.329 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T14:11:42.359 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T14:11:42.359 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22138.log 2024-09-16T14:11:42.360 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22243.log 2024-09-16T14:11:42.360 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T14:11:42.361 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.22138.log: 28.9% -- replaced with /var/log/ceph/ceph-client.admin.22138.log.gz 2024-09-16T14:11:42.361 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.22243.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T14:11:42.362 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22243.log.gz 2024-09-16T14:11:42.362 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T14:11:42.362 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22324.log 2024-09-16T14:11:42.363 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22216.log 2024-09-16T14:11:42.364 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46474.log 2024-09-16T14:11:42.364 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38184.log 2024-09-16T14:11:42.365 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.22324.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T14:11:42.365 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22324.log.gz 2024-09-16T14:11:42.366 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22271.log 2024-09-16T14:11:42.366 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.22216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22216.log.gz 2024-09-16T14:11:42.366 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22163.log 2024-09-16T14:11:42.366 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22299.log 2024-09-16T14:11:42.366 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.22271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22271.log.gz 2024-09-16T14:11:42.367 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.22163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22163.log.gz 2024-09-16T14:11:42.367 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T14:11:42.371 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38748.log 2024-09-16T14:11:42.371 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46474.log.gz 2024-09-16T14:11:42.371 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48611.log 2024-09-16T14:11:42.371 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38184.log.gz 2024-09-16T14:11:42.371 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37031.log 2024-09-16T14:11:42.371 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38748.log.gz 2024-09-16T14:11:42.371 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36128.log 2024-09-16T14:11:42.371 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48611.log.gz 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47241.log 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37031.log.gz 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26009.log 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36128.log.gz 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.47241.log: -- /var/log/ceph/ceph-client.admin.50211.log 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47241.log.gz 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28618.log 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26009.log.gz 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50205.log 2024-09-16T14:11:42.372 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50211.log.gz 2024-09-16T14:11:42.373 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29253.log 2024-09-16T14:11:42.373 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28618.log.gz 2024-09-16T14:11:42.373 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47267.log 2024-09-16T14:11:42.373 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50205.log.gz 2024-09-16T14:11:42.373 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 /var/log/ceph/ceph-client.admin.29253.log: --verbose -- /var/log/ceph/ceph-client.admin.51090.log 2024-09-16T14:11:42.373 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29253.log.gz 2024-09-16T14:11:42.377 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22189.log 2024-09-16T14:11:42.377 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.22299.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T14:11:42.377 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22299.log.gz 2024-09-16T14:11:42.377 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T14:11:42.378 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.22189.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T14:11:42.378 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22189.log.gz 2024-09-16T14:11:42.378 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-osd.7.log: 84.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T14:11:42.378 INFO:teuthology.orchestra.run.smithi164.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T14:11:42.384 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47267.log: /var/log/ceph/ceph-client.admin.27749.log 2024-09-16T14:11:42.384 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47267.log.gz 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46054.log 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51090.log.gz 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.27749.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37643.log 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27749.log.gz 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46054.log.gz 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37551.log 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37643.log.gz 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2024-09-16T14:11:42.385 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37551.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42645.log 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37551.log.gz 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29547.log 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37959.log 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/objectstore_tool.42645.log: /var/log/ceph/ceph-client.admin.29547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37358.log 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29547.log.gz 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49276.log 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37959.log.gz 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50212.log 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37358.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37358.log.gz/var/log/ceph/ceph-client.admin.26282.log 2024-09-16T14:11:42.386 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49276.log: /var/log/ceph/ceph-client.admin.50212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49276.log.gz 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25982.log 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50212.log.gz 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26282.log.gz 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.25982.log: 0.0%/var/log/ceph/ceph-client.admin.37316.log: -- replaced with /var/log/ceph/ceph-client.admin.25982.log.gz 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr:gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz-5 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr:--verbose -- /var/log/ceph/ceph-client.admin.32893.log 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.49408.log: -- /var/log/ceph/ceph-client.admin.46443.log 2024-09-16T14:11:42.387 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29346.log 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32893.log.gz 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26819.log 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26522.log 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46443.log.gz 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29346.log.gz 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26792.log 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26819.log.gz 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26522.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26522.log.gz-5 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr:--verbose -- /var/log/ceph/ceph-client.admin.27222.log 2024-09-16T14:11:42.388 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26792.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.29530.log 2024-09-16T14:11:42.389 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26792.log.gz 2024-09-16T14:11:42.389 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25936.log 2024-09-16T14:11:42.389 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27222.log: /var/log/ceph/objectstore_tool.29530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27222.log.gz 2024-09-16T14:11:42.415 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40524.log 2024-09-16T14:11:42.415 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.25936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38253.log 2024-09-16T14:11:42.415 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25936.log.gz 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31432.log 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.40524.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40524.log.gz 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.38253.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26291.log 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38253.log.gz 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26846.log 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31432.log.gz 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.42645.log.gz 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36082.log 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26291.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.49187.log 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26291.log.gz 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26846.log: /var/log/ceph/ceph-client.admin.36082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26846.log.gz 2024-09-16T14:11:42.416 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49784.log 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36082.log.gz 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28474.log 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49187.log.gz 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27109.log 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49784.log.gz 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49435.log 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28474.log.gz 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26506.log 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27109.log.gz 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36986.log 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49435.log.gz 2024-09-16T14:11:42.417 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool..log 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26506.log.gz 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47125.log 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.36986.log: /var/log/ceph/objectstore_tool..log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36986.log.gz 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27681.log 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr: 95.4% -- replaced with /var/log/ceph/objectstore_tool.29530.log.gz 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39640.log 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47125.log.gz 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48406.log 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27681.log.gz 2024-09-16T14:11:42.418 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.39640.log: -- /var/log/ceph/ceph-client.admin.39639.log 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39640.log.gz 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50947.log 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48406.log.gz 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48653.log 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39639.log: /var/log/ceph/ceph-client.admin.50947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39639.log.gz 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50947.log.gz 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29040.log 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35570.log 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48653.log.gz 2024-09-16T14:11:42.419 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37249.log 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29040.log.gz 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28242.log 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35570.log.gz 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51117.log 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37249.log.gz 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.40975.log 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28242.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28242.log.gz 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28036.log 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51117.log.gz 2024-09-16T14:11:42.420 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/objectstore_tool.40975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28470.log 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26281.log 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28036.log.gz 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46647.log 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28470.log.gz 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39220.log 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26281.log: /var/log/ceph/ceph-client.admin.46647.log: 0.0%gzip -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.26281.log.gz/var/log/ceph/objectstore_tool.32045.log 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46647.log.gz 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50407.log 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39220.log.gz 2024-09-16T14:11:42.421 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 /var/log/ceph/objectstore_tool.32045.log: --verbose -- /var/log/ceph/ceph-client.admin.22028.log 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50407.log.gz 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49651.log 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.22028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22028.log.gz 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49732.log 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49651.log: /var/log/ceph/ceph-client.admin.28139.log 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49651.log.gz 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28926.log 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49732.log.gz 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27826.log 2024-09-16T14:11:42.422 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28139.log.gz 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42210.log 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28926.log.gz 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27826.log.gz 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.40975.log.gz 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr: 95.4% -- replaced with /var/log/ceph/objectstore_tool.32045.log.gz 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27446.log 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37513.log 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42210.log.gz 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48986.log 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27446.log.gz 2024-09-16T14:11:42.423 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26036.log 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37513.log: /var/log/ceph/ceph-client.admin.48986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37513.log.gz 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37770.log 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48986.log.gz 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50201.log 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26036.log.gz 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50542.log 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37770.log.gz 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50785.log 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50201.log.gz 2024-09-16T14:11:42.424 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32677.log 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50542.log.gz 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46177.log 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50785.log.gz 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48847.log 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32677.log.gz 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37434.log 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46177.log.gz 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47416.log 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48847.log.gz 2024-09-16T14:11:42.425 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35380.log 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37434.log.gz 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37669.log 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47416.log.gz 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49981.log 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35380.log.gz 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26301.log 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37669.log.gz 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50812.log 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49981.log.gz 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.26301.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25962.log 2024-09-16T14:11:42.426 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26301.log.gz 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26080.log 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50812.log.gz 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50677.log 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.25962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25962.log.gz 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.26080.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37761.log 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26080.log.gz 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50241.log 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50677.log.gz 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35466.log 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37761.log.gz 2024-09-16T14:11:42.427 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.26939.log 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50241.log.gz 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35466.log.gz 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38826.log 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.0.26939.log: gzip /var/log/ceph/ceph-mon.c.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39741.log 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49076.log 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38826.log.gz 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39741.log: 95.5% -- replaced with /var/log/ceph/ceph-client.0.26939.log.gz 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39741.log.gz 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49114.log 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49076.log.gz 2024-09-16T14:11:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39195.log 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.audit.log: gzip /var/log/ceph/ceph-client.admin.49114.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49114.log.gz 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39195.log.gz 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29104.log 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38505.log 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr: 93.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50204.log 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29104.log.gz 2024-09-16T14:11:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38505.log.gz 2024-09-16T14:11:42.487 INFO:teuthology.orchestra.run.smithi164.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T14:11:42.502 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49705.log 2024-09-16T14:11:42.502 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29310.log 2024-09-16T14:11:42.502 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2024-09-16T14:11:42.502 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49705.log.gz 2024-09-16T14:11:42.502 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25910.log 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27932.log 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29310.log.gz 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44286.log 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.25910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25910.log.gz 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38377.log 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27932.log.gz 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39308.log 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44286.log.gz 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46887.log 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38377.log.gz 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48605.log 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39308.log.gz 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26495.log 2024-09-16T14:11:42.503 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46887.log.gz 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26299.log 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48605.log: 28.2% -- replaced with /var/log/ceph/ceph-client.admin.48605.log.gz 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48803.log 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26495.log.gz 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45937.log 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26299.log.gz 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr: 95.4% -- replaced with /var/log/ceph/objectstore_tool..log.gzgzip 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.49778.log 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48803.log.gz 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28929.log 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.45937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45937.log.gz 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.49778.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49191.log 2024-09-16T14:11:42.504 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49778.log.gz 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28929.log.gz 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38311.log 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49191.log.gz 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32454.log 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29351.log 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38311.log.gz 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40041.log 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.32454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32454.log.gz 2024-09-16T14:11:42.505 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49780.log 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29351.log.gz 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40041.log: /var/log/ceph/ceph-client.admin.48476.log 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40041.log.gz 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27581.log 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49780.log.gz 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29131.log 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48476.log.gz 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26873.log 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27581.log: /var/log/ceph/ceph-client.admin.29131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38106.log 2024-09-16T14:11:42.506 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27581.log.gz 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29131.log.gz 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26090.log 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26873.log.gz 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35571.log 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38106.log.gz 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29197.log 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26090.log.gz 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27707.log 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35571.log.gz 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28010.log 2024-09-16T14:11:42.507 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29197.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29197.log.gz 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43403.log 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27707.log.gz 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28010.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.28010.log.gz-5 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:--verbose -- /var/log/ceph/ceph-client.admin.39945.log 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49489.log 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43403.log.gz 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26514.log 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39945.log.gz 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39505.log 2024-09-16T14:11:42.508 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49489.log.gz 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26088.log 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26514.log.gz 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39505.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41788.log 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39505.log.gz 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51142.log 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26088.log.gz 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/objectstore_tool.41788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38111.log 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37088.log 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51142.log.gz 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38053.log 2024-09-16T14:11:42.509 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38111.log.gz 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37088.log.gz 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43482.log 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38053.log: /var/log/ceph/ceph-client.admin.48760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29341.log 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38053.log.gz 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.43482.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26944.log 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43482.log.gz 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40746.log 2024-09-16T14:11:42.510 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29341.log.gz 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26091.log 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26944.log.gz 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.40746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40746.log.gz 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49381.log 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26711.log 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26091.log.gz 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49381.log.gz 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49354.log 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26711.log.gz 2024-09-16T14:11:42.511 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49624.log 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49354.log.gz 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51001.log 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49624.log: /var/log/ceph/ceph-client.admin.51001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28305.log 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51001.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28143.log 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49516.log 2024-09-16T14:11:42.512 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28305.log.gz 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46711.log 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33943.log 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28143.log.gz/var/log/ceph/ceph-client.admin.49516.log: 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49516.log.gz 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28723.log 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46711.log.gz 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46402.log 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.41788.log.gz 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33943.log.gz 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50569.log 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28723.log.gz 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49992.log 2024-09-16T14:11:42.513 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46402.log.gz 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50569.log.gz 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49570.log 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49976.log 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49992.log.gz 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29443.log 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49570.log.gz 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49991.log 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49976.log.gz 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38551.log 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29443.log.gz 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48927.log 2024-09-16T14:11:42.514 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49991.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49991.log.gz 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.38551.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38621.log 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38551.log.gz 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51028.log 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48927.log.gz 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28342.log 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38621.log.gz/var/log/ceph/ceph-client.admin.51028.log: 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50623.log 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51028.log.gz 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26507.log 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28342.log.gz 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.50623.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26077.log 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50623.log.gz 2024-09-16T14:11:42.515 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50215.log 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26507.log.gz 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27552.log 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26077.log.gz 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46059.log 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50215.log.gz 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40078.log 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27552.log.gz 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50758.log 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46059.log.gz 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.40078.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49783.log 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40078.log.gz 2024-09-16T14:11:42.516 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26765.log 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50758.log.gz 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26499.log 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49783.log.gz 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27354.log 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26765.log.gz 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41792.log 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26499.log.gz 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22194.log 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27354.log.gz 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26600.log 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41792.log.gz 2024-09-16T14:11:42.517 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28759.log 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.22194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22194.log.gz 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50003.log 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26600.log.gz 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28852.log 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28759.log.gz 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34533.log 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50003.log.gz 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27281.log 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28852.log.gz 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50515.log 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34533.log.gz 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35811.log 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27281.log.gz 2024-09-16T14:11:42.518 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27406.log 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50515.log.gz 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35354.log 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35811.log.gz 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35971.log 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27406.log.gz 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50974.log 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35354.log.gz 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28857.log 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35971.log.gz 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26521.log 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50974.log.gz 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26948.log 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28857.log.gz 2024-09-16T14:11:42.519 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38696.log 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26521.log.gz 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46152.log 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26948.log.gz 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26087.log 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38696.log.gz 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28431.log 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46152.log.gz 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49782.log 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26087.log.gz 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43404.log 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28431.log: 0.0%/var/log/ceph/ceph-client.admin.49782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27189.log 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28431.log.gz 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49782.log.gz 2024-09-16T14:11:42.520 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.40030.log 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43404.log.gz 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37186.log 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27189.log.gz 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/objectstore_tool.40030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29485.log 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48300.log 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37186.log: 0.0%/var/log/ceph/ceph-client.admin.29485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29485.log.gz 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39054.log 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48300.log.gz 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37186.log.gz 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46997.log 2024-09-16T14:11:42.521 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39054.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45334.log 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39054.log.gz 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38432.log 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46997.log.gz 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.45334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr: 95.5% -- replaced with /var/log/ceph/objectstore_tool.40030.log.gz 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35645.log 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49785.log 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38432.log.gz 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39392.log 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35645.log.gz 2024-09-16T14:11:42.522 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49327.log 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49785.log.gz 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.39392.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39532.log 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39392.log.gz 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35755.log 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49327.log.gz 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47388.log 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39532.log.gz 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39153.log 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35755.log.gz 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47388.log.gz 2024-09-16T14:11:42.523 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49244.log 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39153.log.gz 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35496.log 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26454.log 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49244.log.gz 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35496.log.gz 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27493.log 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26454.log.gz 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.35697.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28540.log 2024-09-16T14:11:42.524 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log /var/log/ceph/ceph-client.admin.27493.log: 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27493.log.gz 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47073.log 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28540.log.gz 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22327.log 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50866.log 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47073.log.gz 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.22327.log: -- /var/log/ceph/ceph-client.admin.26905.log 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22327.log.gz 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50866.log.gz 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26076.log 2024-09-16T14:11:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39024.log 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26905.log.gz 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47160.log 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26076.log.gz 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39024.log.gz 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.47160.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46291.log 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47160.log.gz 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46360.log 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.46291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38558.log 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46291.log.gz 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39807.log 2024-09-16T14:11:42.526 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46360.log.gz 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38941.log 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38558.log.gz 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33499.log 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26738.log 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0%/var/log/ceph/ceph-client.admin.38941.log: -- replaced with /var/log/ceph/ceph-client.admin.39807.log.gz 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38941.log.gz 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.33499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33499.log.gz 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26287.log 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26738.log.gz 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39919.log 2024-09-16T14:11:42.527 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26287.log.gz 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50650.log 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27147.log 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39919.log.gz 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49995.log 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50461.log 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27147.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27147.log.gz 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50650.log.gz 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.49995.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38657.log 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49995.log.gz 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51055.log 2024-09-16T14:11:42.528 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50461.log.gz 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26952.log 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38657.log.gz 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29135.log 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51055.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39480.log 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51055.log.gz 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26952.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25836.log 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26952.log.gz 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29135.log.gz 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39480.log.gz 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29009.log 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.25836.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50893.log 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25836.log.gz 2024-09-16T14:11:42.529 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47894.log 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29009.log.gz 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37487.log 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50893.log.gz 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47894.log.gz 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50002.log 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50191.log 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37487.log.gz 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29762.log 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50002.log.gz 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17216.log 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50191.log.gz 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.29762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29762.log.gz 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45994.log 2024-09-16T14:11:42.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49462.log 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.tmp-client.admin.17216.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17216.log.gz 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.45994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45994.log.gz 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46546.log 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38872.log 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49462.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46748.log 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46546.log.gz 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50488.log 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38872.log.gz 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50839.log 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46748.log.gz/var/log/ceph/ceph-client.admin.50488.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T14:11:42.531 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50488.log.gz 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38459.log 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35946.log 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50839.log.gz/var/log/ceph/ceph-osd.3.log: 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38459.log.gz 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46831.log 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35946.log.gz 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28567.log 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46831.log.gz 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49543.log 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:gzip /var/log/ceph/ceph-client.admin.28567.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31214.log 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28567.log.gz/var/log/ceph/ceph-client.admin.49543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39839.log 2024-09-16T14:11:42.532 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49543.log.gz 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.31214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50596.log 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31214.log.gz 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46246.log 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.39839.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39839.log.gz 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50596.log.gz 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50731.log 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50704.log 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46246.log.gz 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38183.log 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50731.log.gz 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50704.log.gz 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49597.log 2024-09-16T14:11:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49678.log 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38183.log.gz 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26300.log 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49597.log.gz/var/log/ceph/ceph-client.admin.49678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48512.log 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49678.log.gz 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49786.log 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26300.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31237.log 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26300.log.gz 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48512.log: 0.0%/var/log/ceph/ceph-client.admin.49786.log: -- replaced with /var/log/ceph/ceph-client.admin.48512.log.gz 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27067.log 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49786.log.gz 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46830.log 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/objectstore_tool.31237.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.30426.log 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.27067.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27067.log.gz 2024-09-16T14:11:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46830.log.gz 2024-09-16T14:11:42.535 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37160.log 2024-09-16T14:11:42.535 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28653.log 2024-09-16T14:11:42.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/objectstore_tool.30426.log: /var/log/ceph/ceph-client.admin.37160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37160.log.gz 2024-09-16T14:11:42.540 INFO:teuthology.orchestra.run.smithi079.stderr: 95.6% -- replaced with /var/log/ceph/objectstore_tool.31237.log.gz 2024-09-16T14:11:42.540 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35670.log 2024-09-16T14:11:42.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.28653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46550.log 2024-09-16T14:11:42.540 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28653.log.gz 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41175.log 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35670.log.gz 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46550.log.gz 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48931.log 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41175.log.gz 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26089.log 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48931.log.gz 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49277.log 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.26089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50434.log 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26089.log.gz 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49277.log: 95.4% -- replaced with /var/log/ceph/objectstore_tool.30426.log.gz 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49277.log.gz 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38315.log 2024-09-16T14:11:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50434.log.gz 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27879.log 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.38315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38315.log.gz 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.27879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27879.log.gz 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37997.log 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.35837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30203.log 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.37997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49787.log 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37997.log.gz 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.30203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30203.log.gz 2024-09-16T14:11:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49787.log.gz 2024-09-16T14:11:42.674 INFO:teuthology.orchestra.run.smithi079.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T14:11:42.804 INFO:teuthology.orchestra.run.smithi079.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T14:11:43.266 INFO:teuthology.orchestra.run.smithi164.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T14:11:44.744 INFO:teuthology.orchestra.run.smithi164.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T14:11:44.928 INFO:teuthology.orchestra.run.smithi079.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T14:11:45.190 INFO:teuthology.orchestra.run.smithi164.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T14:11:46.130 INFO:teuthology.orchestra.run.smithi164.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T14:11:46.131 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-09-16T14:11:46.131 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m3.797s 2024-09-16T14:11:46.131 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m9.696s 2024-09-16T14:11:46.131 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.356s 2024-09-16T14:11:46.387 INFO:teuthology.orchestra.run.smithi079.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T14:11:47.561 INFO:teuthology.orchestra.run.smithi079.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T14:11:47.837 INFO:teuthology.orchestra.run.smithi079.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T14:11:47.838 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-09-16T14:11:47.838 INFO:teuthology.orchestra.run.smithi079.stderr:real 0m5.506s 2024-09-16T14:11:47.838 INFO:teuthology.orchestra.run.smithi079.stderr:user 0m17.327s 2024-09-16T14:11:47.838 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m0.699s 2024-09-16T14:11:47.839 INFO:tasks.ceph:Archiving logs... 2024-09-16T14:11:47.839 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/remote/smithi079/log 2024-09-16T14:11:47.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T14:11:50.312 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/remote/smithi164/log 2024-09-16T14:11:50.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T14:11:51.696 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T14:11:51.705 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T14:11:51.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T14:11:51.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T14:11:51.776 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T14:11:51.784 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T14:11:51.784 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:11:51.787 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:11:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T14:11:51.807 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================== 2024-09-16T14:11:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: hv01.front.sepi 67.205.162.81 3 u 64 64 177 0.095 0.008 0.028 2024-09-16T14:11:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: hv02.front.sepi 63.231.80.2 3 u 62 64 177 0.096 0.160 0.115 2024-09-16T14:11:51.807 INFO:teuthology.orchestra.run.smithi079.stdout:*hv03.front.sepi 74.6.168.72 3 u 4 64 377 0.075 -0.147 0.055 2024-09-16T14:11:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:11:51.809 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T14:11:51.809 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2024-09-16T14:11:51.809 INFO:teuthology.orchestra.run.smithi164.stdout:+hv01.front.sepi 67.205.162.81 3 u 6 64 377 0.092 0.232 0.553 2024-09-16T14:11:51.810 INFO:teuthology.orchestra.run.smithi164.stdout:+hv02.front.sepi 63.231.80.2 3 u 60 64 177 0.071 0.714 0.392 2024-09-16T14:11:51.810 INFO:teuthology.orchestra.run.smithi164.stdout:*hv03.front.sepi 74.6.168.72 3 u 4 64 377 0.084 0.093 0.544 2024-09-16T14:11:51.810 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T14:11:51.811 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T14:11:51.864 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T14:11:51.864 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T14:11:51.901 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T14:11:51.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T14:11:51.969 INFO:teuthology.task.internal:Duration was 804.476732 seconds 2024-09-16T14:11:51.969 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T14:11:52.002 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T14:11:52.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T14:11:52.005 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T14:11:52.055 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T14:11:52.055 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-09-16T14:11:52.055 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T14:11:52.066 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T14:11:52.066 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T14:11:52.079 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T14:11:52.080 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:11:52.082 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:11:52.118 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T14:11:52.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T14:11:52.120 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T14:11:52.204 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T14:11:52.214 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T14:11:52.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T14:11:52.227 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T14:11:52.242 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T14:11:52.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:11:52.253 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:11:52.264 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-09-16T14:11:52.268 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-16T14:11:52.295 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:11:52.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:11:52.342 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:11:52.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:11:52.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T14:11:52.360 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T14:11:52.361 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/remote/smithi079 2024-09-16T14:11:52.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T14:11:52.394 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907316/remote/smithi164 2024-09-16T14:11:52.395 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T14:11:52.419 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T14:11:52.419 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T14:11:52.421 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T14:11:52.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T14:11:52.436 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T14:11:52.437 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T14:11:52.461 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T14:11:52.461 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T14:11:52.467 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T14:11:52.472 INFO:teuthology.orchestra.run.smithi079.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 14:11 /home/ubuntu/cephtest 2024-09-16T14:11:52.475 INFO:teuthology.orchestra.run.smithi164.stdout: 1048614 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 14:11 /home/ubuntu/cephtest 2024-09-16T14:11:52.477 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T14:11:52.538 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T14:11:52.614 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read} duration: 804.4767324924469 flavor: default owner: scheduled_sangadi@teuthology success: true 2024-09-16T14:11:52.614 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T14:11:52.695 INFO:teuthology.run:pass